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>J73AVS  also  provides  a  large  number  of  source  analysis  reports,  ranging  from  symbol  cross 
references  and  descriptions  of  symbol  usages  to  procedure  calling  trees  and  descriptions 
of  all  procedures  being  analyze^,  These  reports  are <tvary^ useful  for  original  code 
development,  rehost ing  "foreign1!1  code,  debugging  and  testing,  code  enhancement,  and  code 
maintenance, 

.  y. 

This  report^describes  the  features  of  J73AVS  in  a  brief  manner ,  the  history  of  the  project, 
documents  and  technical  papers  resulting  from  the  project,  and  specific  details  on  how  to 
obtain  the  J73AVS  software  and  selected  project  reports.  Also  included  are  implications  on 
further  research  in  software  engineering  as  it  pertains  jo  J73AVS. 
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1  INTRODUCTION 

The  JOVIAL  J73  Automated  Verification  System  was  developed  under 
sponsorship  by  the  Rome  Air  Development  Center  (RADC)  at  Grlffiss  Air  Force 
Base,  NY,  under  Contract  F30602-79-C-0265. 

The  resulting  tool,  J73AVS,  is  an  interactive  computer  program  that 
analyzes  any  source  code  written  in  JOVIAL  J73.  The  primary  objective  of 
J73AVS  is  to  provide  static  and  dynamic  testing  assistance.  To  do  this, 

J73AVS  detects  a  variety  of  semantic  and  data  flow  errors,  reports  execution 
coverage  (l.e.,  frequency  of  statements,  control  branches,  and  procedures 
executed  by  each  test  case),  reports  execution  timing  (in  terms  of  CPU 
milliseconds)  for  procedures  or  certain  user-designated  portions  of  the  JOVIAL 
J73  program,  reports  execution  tracing  (ordering)  of  control  branches  or 
procedures,  and  keeps  track  of  the  test  coverage  history  from  run  to  run.  To 
assist  with  functional  testing  of  programs,  J73AVS  provides  a  local  assertion 
construct  which  triggers  an  output  message  when  it  is  violated. 

J73AVS  also  provides  a  large  number  of  source  analysis  reports,  ranging 
from  symbol  cross  references  and  descriptions  of  symbol  usages  to  procedure 
calling  trees  and  descriptions  of  all  procedures  being  analyzed.  These 
reports  are  very  useful  for  original  code  development,  rehosting  “foreign** 
code,  debugging  and  testing,  code  enhancement,  and  code  maintenance. 

J73AVS  features  and  its  ease  of  operation  are  described  more  fully  in 
Sec.  2.  Since  the  contract  has  had  two  major  phases  (test  tool  study  and  tool 
Implementation),  the  project  history  is  covered  in  Sec.  3.  J73AVS  avail¬ 
ability  and  installation  configuration  are  discussed  in  Sec.  4.  Each  deliver¬ 
able  document  is  briefly  described  in  Sec.  5  along  with  information  concerning 
how  to  obtain  each  report. 

Following  the  overview  of  J73AVS  capabilities  in  Sec.  2  is  a  discussion 
in  Sec.  6  of  possible  future  research  regarding  J73AVS.  Three  conference 
papers  resulted  from  the  development  of  J73AVS.  These  are  reprinted  in 
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Appendix  A.  Terms  and  abbreviations  common  to  J73AVS  are  provided  in  Appendix 
B.  J73AVS  operates  interactively,  driven  by  simple  user  commands.  The 
commands  that  direct  J73AVS  to  either  perform  some  activity  or  to  report 
specific  results  are  given  in  Appendix  C. 
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OVERVIEW  OF  CAPABILITIES 

J73AVS  can  operate  in  either  batch  or  interactive  mode.  The  user 
directs  J73AVS  through  a  command  language;  each  major  processing  activity  has 
a  command  keyword  associated  with  it  and  command  parameters  are  used  to 
specify  processing  or  reporting  options.  The  type  of  processing  and  its 
general  sequence  are  shown  in  Fig.  2.1.  While,  the  flowchart  in  Fig.  2.1  shows 
a  number  of  processing  activities,  J73AVS  performs  a  number  of  them  collec¬ 
tively  when  a  user  gives  a  J73AVS  command.  Prompts  are  output  by  J73AVS  if 
commands  are  incorrect,  out  of  order,  or  prerequisite  information  is  needed. 
J73AVS  process  and  reporting  commands  are  given  in  App.  C. 

2.1  J73AVS  OPERATION 

One  or  more  JOVIAL  J73  source  modules  (compilation  units)  or  J73  COPY 
texts  are  supplied  to  J73AVS  as  input.  There  are  no  restrictions  on  the 
length  of  modules.  Up  to  250  modules  can  be  stored  on  the  J73AVS  database. 

The  source  text  must  be  in  the  same  form  as  if  it  were  being  input  to  the 
JOVIAL  J73  compiler.  Job  control  setups  are  provided  in  the  User's  Manual 
(Appendix  B  for  the  IBM  and  in  Appendix  C  for  the  VAX). 

The  source  text  for  an  entire  program  need  not  be  supplied  to  J73AVS  for 
static  and  data  flow  processing  and  for  procuring  automated  reports.  Some 
modules  may  be  supplied  as  stubs  (e.g.,  module  heading,  possibly  some  logical 
assertions  describing  the  function,  and  a  return)  or  not  at  all.  J73AVS 
builds  a  database  of  module  interface  information,  which  can  be  saved  and 
re-used  from  run-to-run  during  program  development  or  testing.  Therefore, 
J73AVS  can  be  used  to  incrementally  build  and  test  software.  Updated  modules 
automatically  replace  old  ones  on  the  J73AVS  database.  This  database  is 
highly  compressed,  even  though  it  contains  a  wealth  of  information  about  the 
user '8  source  code.  The  saved  database  is  approximately  three  times  the  size 
of  the  user's  source  code  file. 

Execution  analysis  by  J73AVS  requires  instrumenting  the  source  text, 
which  J73AVS  performs  automatically  when  the  user  requests  it.  Anywhere  from 
one  J73  procedure  to  all  the  source  modules  available  on  the  database  can  be 
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Figure  2.1.  Overview  of  J73AVS 
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instrumented.  J73AVS  outputs  the  instrumented  source  as  a  complete  module, 
ready  for  input  to  a  JOVIAL  J73  compiler.  To  obtain  J73AVS  output  for 
assertion  violations,  timing,  coverage,  or  tracing,  the  program  (with  some  or 
all  modules  instrumented)  is  executed  with  test  data  supplied  by  the  user. 
Programs  instrumented  for  tracing  or  assertions  will  produce  short  messages 
from  J73AVS,  interspersed  with  the  program's  own  output,  during  execution. 

For  coverage  and  timing  analysis,  program  execution  in  the  J73AVS  environment 
differs  from  normal  program  execution  in  the  following  ways:  (1)  one  or  more 
modules  have  been  instrumented  before  compilation,  (2)  a  small  J73AVS  data 
collection  routine  is  loaded  before  execution,  and  (3)  a  J73AVS  audit  file  is 
written  during  execution.  This  file  is  used  by  J73AVS  to  produce  reports  for 
coverage  and  timing  analysis.  The  J73AVS  database  is  not  used  during  the 
program's  execution.  These  major  functions  and  data  files  are  illustrated  in 
Fig.  2.2. 


Although  J73AVS  exists  as  a  single  program,  it  is  best  considered  as  a 
collection  of  tools  that  interact  with  the  user.  Some  of  the  facilities,  such 
as  automated  documentation,  static  error  reporting,  and  instrumentation  are 
completely  automated  and  require  only  that  the  user  initiate  the  tasks.  Other 
processes,  such  as  execution-time  data  collection  or  retesting  assistance, 
require  more  information.  The  user  must  provide  test  data  and  select  the  test 
targets . 

J73AVS  provides  detailed  information  about  the  static  and  execution-time 
behavior  of  the  program.  The  user  directs  the  processing  performed  by  J73AVS, 
analyzes  the  output  produced  by  J73AVS,  and  determines  subsequent  action. 

2.2  ROLE  IN  THE  SOFTWARE  DEVELOPMENT  CYCLE 

The  role  of  J73AVS  in  the  software  development  cycle  is  to  provide 
automated  assistance  wherever  possible  during  program  development:  coding, 
debugging,  testing,  maintenance,  and  retesting.  J73AVS  users  play  an  active 
part  in  the  cycle,  as  shown  in  Fig.  2.3.  This  figure  breaks  up  the  develop¬ 
ment  cycle  and  shows  the  flow  of  control  and  information  between  J73AVS  and 
the  user. 
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Using  Fig.  2.3  as  a  basis,  a  typical  sequence  of  J73AVS-supported 
operations  is: 

1.  JOVIAL  J73  source  text,  perhaps  with  assertions,  is  read  by  J73AVS 
as  one  or  more  compilable  modules. 

2.  J73AVS  produces  program  analysis  reports  showing  control  struc¬ 
ture,  symbol  usage,  calling  hierarchy,  etc.,  as  well  as  a  static 
analysis  report  showing  errors  and  dangerous  programming  prac¬ 
tices. 

3.  Using  the  reports  as  a  guide,  the  source  modules  are  changed  or 
new  modules  are  added  to  the  program. 

4.  J73AVS  reports  the  interaction  of  the  new  or  changed  modules  with 
the  rest  of  the  program.  This  information,  in  turn,  may  show  the 
need  to  modify  other  modules. 
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Figure  2.3.  Role  of  J73AVS  in  the  Software  Development  Cycle 


For  debugging,  the  program  is  instrumented  by  J73AVS  and  executed 
with  an  initial  test  case  supplied  by  the  user. 

Assertion  messages,  variable  and  module  tracing,  and  execution 
timing  reports  can  be  used  for  debugging. 

Using  the  J73AVS  reports,  the  user  chooses  to  create  more  test 
data  or  instrument  other  modules. 

For  testing,  the  same  cycle  of  instrumentation  and  execution  is 
repeated,  but  for  a  different  goal:  rather  than  detecting  and 
locating  errors,  testing  aims  to  demonstrate  that  the  entire 
program  has  been  exercised  Co  some  degree.  The  J73AVS  execution 
analysis  reports  show  the  thoroughness  of  execution  coverage. 

The  user  evaluates  execution  coverage  reports,  the  program's  own 
execution  results,  and  the  program  specification  to  determine  if 
testing  is  complete. 

J73AVS  provides  branch  sequence  Information  to  retest  targets 
chosen  by  the  user.  A  test  history  of  execution  coverage  assists 
the  user  in  choosing  targets  for  retesting. 
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Executable  assertions  provide  a  means  for  a  programmer  to  specify 
expected  behavior.  Logical  condition  assertions  can  be  used  for  reporting 
execution-time  exceptions,  stress  testing,  and  manual  or  automated  test  data 
generation.  When  assertions  are  left  as  comments  in  the  source  code  they  can 
be  used  as  inline  documentation  of  the  program's  specifications. 

To  assist  with  reliable  system  development  and  maintenance,  J73AVS 
provides  substantial  program  analysis  reporting  on  structural  hierarchy, 
symbol  usage,  invocations,  certain  J73  constructs,  and  system  characteristics. 

The  user  has  control  over  obtaining  high-  or  low-level  information  through 
the  tool's  command  language. 


Normal  compilation  using  JOVIAL  J73  compilers  can  detect  many  syntax  and 
semantic  errors.  Other  errors,  such  as  uninitialized  variables,  possible 
infinite  loops,  unreachable  code,  certain  improper  constructs,  and  dangerous 
coding  practices  (like  transferring  into  CASE  or  IF  statements)  will  be 
reported  by  J73AVS.  The  user  can  specify  the  degree  of  analysis  to  the  error, 
warning,  or  message  level. 

Debugging  is  also  supported  by  assertion  exceptions,  variable  and  module 
execution  tracing,  and  execution  timing  reports.  When  the  program's  execution 
behavior  deviates  from  the  acceptable  logical  behavior  as  specified  by  the 
assertions,  it  is  immediately  reported  in  the  program's  output.  The  user- 
embedded  assertions  have  no  effect  on  program  control  flow  until  they  are 
violated;  at  that  time  the  violation  is  reported  with  the  source  statement 
number  of  the  assertion. 

Testing 

The  primary  purpose  of  program  coverage  analysis  is  to  provide  a  measure 
of  the  level  of  testing.  This  measuring  technique  uses  the 
program's  control  structure  as  a  guide.  Structure-based  testing  means  that 
the  program's  control  structures  are  analyzed  for  execution  behavior;  that  is. 


whether  the  structures  are  exercised  and  in  what  order.  Structure-based 
testing  can  uncover  errors  due  to  untested  branches  or  improper  sequences  of 
branches.  J73AVS  provides  program  unit  or  branch  tracing  and  analyzes 
execution  coverage  of  program  units,  branches,  and  statements.  Further, 

J73AVS  assembles  the  timing  information  from  program  unit  tracing  and  user- 
directed  timing  probes  into  an  execution  timing  report. 

Retesting  Assistance 

Retesting  software  is  performed  when  analysis  shows  that  prior  testing 
is  inadequate  (insufficient  branch  coverage,  not  all  functions  demonstrated, 
etc.)  or  when  program  changes  have  taken  place.  The  proper  approach  to  take 
in  retesting  is  highly  dependent  upon  the  characteristics  of  the  program  being 
tested  as  well  as  the  measures  being  used  to  evaluate  testing  completeness. 
Section  2  of  the  User's  Manual  provides  a  methodology  for  testing  and  re¬ 
testing  software  for  the  purpose  of  improving  structural-testing  completeness. 

To  determine  the  sequences  of  branches  which  lead  to  an  untested  branch 

or  statement,  the  user  can  request  that  the  “reaching  set”  be  computed  between 

two  specified  statements  (or  from  the  program  unit's  entry).  After  the  flow 
of  control  is  identified  by  J73AVS,  the  user  can  backtrack  through  the  program 

to  the  actual  test  data.  New  test  data  can  be  created  by  using  J73AVS  module 

interaction,  invocation,  and  execution  coverage  reports.  Unfortunately, 
automatic  test  data  generators  which  use  symbolic  execution  are  not  yet 
general  enough,  easy  to  use,  or  reliable.  Therefore,  J73AVS  has  no  test  data 
generation  capability  at  this  time. 


The  testing  history  maintained  by  J73AVS  is  useful  in  attaining  coverage 
testing  goals  and  for  determining  targets  for  retesting.  Program  unit  and 
coverage  information  is  saved  in  a  concise  way  for  each  test  case.  The 
results  of  subsequent  execution  runs  can  be  added,  providing  a  cumulative 
report  of  all  tests. 
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PROJECT  HISTORY 


3.1  PROJECT  GOALS 


The  J73AVS  contract  was  awarded  in  September  1979  during  the  period  of 
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final  JOVIAL  J73  language  definition.  ’  The  primary  goals  of  the  project 
were  to  identify  useful  automated  testing  techniques  for  the  new  dialect  of 
JOVIAL  and  to  develop  an  automated  tool  that  incorporated  the  best  and  most 
feasible  of  the  identified  techniques.  The  intent  was  to  make  such  a  tool 
available  to  Air  Force  contractors  shortly  after  the  JOVIAL  J73  compilers 
became  available.  Therefore,  the  project  was  in  two  parts:  (1)  a  six-month 
study  phase  and  (2)  a  seventeen-month  implementation  phase  (separated  by  a 
short  project  review  period). 


3.2  PROJECT  DELIVERABLES 

The  basic  contract  called  for  delivery  of  J73AVS,  written  in  JOVIAL  J73, 
on  two  computers:  the  ITEL  AS/5-3  (OS/MVS)  at  the  Aeronautical  Systems 
Division  Computer  Center  (ASD/AD)  at  Wright-Patterson  Air  Base  and  the  DEC20 
(TOPS20)  at  RADC.  Modifications  to  the  contract  resulted  in  one  delivery  of 
J73AVS  written  in  structured  FORTRAN  first,  then  in  JOVIAL  J73,  on  the  Amdahl 
470  (TSO,  OS/MVS)  at  ASD/AD,  a  partial  delivery  on  the  DEC20  at  RADC,  and  a 
delivery  in  structured  FORTRAN  on  the  VAX  11/780  (VMS)  at  ASD/AD. 


Two  sets  of  J73AVS  training  classes  were  given  at  ASD/AD:  one  for  the 
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Amdahl  (the  IBM  370-equivalent  version  of  J73AVS)  and  one  for  the  VAX  (the 
VAX  version  of  J73AVS).  Both  fully  delivered  versions  of  J73AVS  underwent 
formal  acceptance  testing.  The  VAX  version  included  a  six-month  maintenance 
period.  Both  versions  of  J73AVS  are  functionally  equivalent. 


MIL-STD-1 589A ,  Military  Standard  JOVIAL  J73,  March  15,  1979. 


MIL-STD- 1 589B ,  Military  Standard  JOVIAL  J73,  June  6,  1980. 


The  Amdahl  computer  at  ASD  was  later  replaced  by  an  NAS  7000  (another  IBM 
370-equivalent  computer). 


3.3  PROJECT  ACTIVITIES 

The  primary  project  activities  can  be  highlighted  as  follows: 

•  Sept.  1979  -  Study  phase  began. 

•  Jan.  1980  -  Preliminary  J73AVS  design  briefing  given  at  the 

JOVIAL  User's  Group.  Solicited  input  on  desir¬ 
able  J73AVS  capabilities. 

•  March  1980  -  Study  phase  reports*  delivered: 

1 .  Functional  Description 

2.  System/ Subsystem  Specification 

3.  Final  Report 

•  May  1980  -  Implementation  phase  began. 

•  July  1980  -  Gave  status  briefing  at  the  JOVIAL  User's  Group. 

Solicited  input  on  what  J73AVS  analysis  is 
desired  for  the  J73  DEFINE  construct  and  how  J73 
programmers  plan  to  handle  input  and  output. 

•  August  1980  -  Contract  modified  to  add  J73AVS  analysis  of 

MIL-STD-1 589B. 

•  October  1981  -  Test  plan  delivered  for  the  Amdahl  host. 

•  Nov.  1981  -  Formal  acceptance  testing  and  training  course 

given  on  the  Amdahl  at  ASD/ADOL.  User's  Manual 
delivered. 

•  June  1962  -  Contract  was  modified  to  translate  J73AVS  from 

structured  FORTRAN  to  JOVIAL  J73  on  the  Amdahl 
and  complete  a  data  flow  analysis  capability. 

•  Sept.  1982  -  Contract  modified  to  add  structured  FORTRAN 

version  of  J73AVS  on  the  VAX  11/780,  along  with 
six  months  of  maintenance. 

*  All  reports  are  listed  in  Sec.  5. 
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Nov.  1982 


Formal  acceptance  testing  on  the  J73  Amdahl 
version  of  J73AVS.  DEC20  version  partially 
installed. 


•  Dec.  1982 


•  March  1983 


•  May  1983 

•  Sept.  1983 


Program  Specification  and  Program  Maintenance 
manuals  delivered.  Final  Amdahl  J73AVS  User's 
Manual  delivered. 

VAX  J73AVS  installation  completed.  Acceptance 
testing  and  training  performed  at  ASD/AD.  New 
User's  Manuals  issued  containing  both  IBM- 
equivalent  and  VAX  operating  instructions  for 
J73AVS. 

VAX  J73AVS  Program  Maintenance  Manual  delivered. 

End  of  contract.  Final  VAX  and  Amdahl  versions 
delivered  to  ASD/ADOL.  Final  Report  delivered. 
Final  Program  Specification  delivered. 


3.4  PROJECT  ADMINISTRATION 

The  contract  has  been  sponsored  and  administered  by  RADC,  with  project 
monitoring  performed  by  Frank  S.  LaMonica.  VAX  rehosting  and  maintenance  were 
funded  by  the  Embedded  Computer  Standardization  Program  Office  (ASD/AXS). 


4  J73AVS  AVAILABILITY 

J73AVS  is  available  in  source  or  binary  object  form  from  the  Language 
Control  Facility  (LCF)  at  Wright-Patterson  Air  Force  Base.  The  J73AVS  User's 
Manual,  Program  Maintenance  Manual  (both  described  in  Section  5),  and  a 
version  description  document  are  also  available  through  the  LCF.  J73AVS 
software  is  distributed  on  9-track,  1600  bits-per-inch  magnetic  tape.  The  LCF 
point  of  contact  is: 

Georgeanne  Chitwood 
ASD/ADOL 

WPAFB,  Ohio  45433 

513-255-4472 

AV785-4472 

The  LCF  can  provide  distribution  tapes  for  either  the  VAX  or  IBM 
370-equlvalent  versions  of  J73AVS.  A  summary  of  each  installation  configura¬ 
tion  is  provided  in  Table  4.1  for  the  VAX  and  Table  4.2  for  the  IBM. 
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TABLE  4.1 

J73AVS  VAX  INSTALLATION  AT  WRIGHT-PATTERSON 


Date 

Version 

Computer 

Operating  System 
Source  Language 
Preprocessor 
Compiler 

Non-Standard  Features 
Configuration 

Memory  Requirements 

Executable  Task  Image  Size 


September  27,  1983 
VF092783 
VAX  11/780 
VMS  3.2 

JTRAN  (Structured  FORTRAN) 

JTRAN  (generates  ANSI  FORTRAN-66) 

VAX-11  FORTRAN  V 3. 3-45 
None 

ANSI  FORTRAN  66 

Internal  J73AVS  assertions  disabled 

Depends  on  host  VAX  system  parameters. 

A  host  with  a  minimum  of  1MB  of  core  is 
recommended  for  reasonable  performance. 

J73AVS.EXE  needs  approximately  800 
disk  blocks 


*>.- 


m. 


Kv 
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TABLE  4.2 

J73AVS  IBM-EQUIVALENT  INSTALLATION  AT  WRIGHT-PATTERSON 


>ate 

Version 

Computer 

Operating  System 
Source  Language 

Preprocessor 

Compilers 

Non-Standard  Features 


Configuration 

Core  Memory  Requirements: 
Executable  Load  Module: 


September  27,  1983 
IBMJ092783 

NAS  7000  (IBM  370  equivalent) 

Multiple  Virtual  System  (MVS)  3.8 

MIL-STD  1589B  JOVIAL/J73 
(utility  routines  in  FORTRAN  and 
assembler) 

None 

JOVIAL/ J7 3  Version  R0201-03.000  (040183) 
FORTRAN  H  Extended 
IBM  Assembler 

FORTRAN  I/O 

BAL  PDS-meraber  access  system  routines 
(provided  by  SofTech,  Inc.) 

JOVIAL  J73  PDS-member  access  utilities 
BAL  CPU-clock  access  routines 

Batch,  Interactive 
Non-over layed 

JOVIAL/J73  code  is  not  optimized 

954K  Bytes  (non-over layed,  non-optlmlzed) 

A790684. J73AVS.LOADMOD(J73AVS) 


5  RELATED  DOCUMENTS 

This  section  contains  descriptive  information  on  each  documentation  item 
that  was  delivered  under  the  contract.  The  format  for  this  information  is  the 
report  title,  abstract  or  purpose  of  the  document,  followed  by  the  report's 
table  of  contents.  For  the  Training  Course  workbook,  the  course  agenda  is 
provided  as  a  description. 
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1.  Title:  JOVIAL  J73  Automated  Verification  System  Functional  Descrip¬ 

tion 

Report  No.:  CR-1-947 


Authors: 

Date: 

Abstract: 


C.  Gannon,  N.B.  Brooks 
March  1980 


This  report  describes  the  functions  of  an  Automated  Verification  System 
(AVS)  for  the  JOVIAL  J73  computer  language,  from  the  point  of  view  of  the 
user.  The  purpose  of  the  AVS  is  to  improve  the  reliability  and  maintain¬ 
ability  of  JOVIAL  J73  software.  The  internal  operations  of  the  AVS  are 
described  in  another  report,  the  System/Subsystem  Specification. 


Table  of  Contents: 
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ABSTRACT 
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1.1  Purpose  of  the  Functional  Description 

1.2  Project  References 

1.3  Terms  and  Abbreviations 
SYSTEM  SUMMARY 

2.1  Background 

2.2  Objectives 

2.3  Existing  Methods  and  Procedures 

2.4  Proposed  Methods  and  Procedures 
DETAILED  CHARACTERISTICS 

3.1  Specific  Performance  Requirements 

3.2  System  Functions 

3.3  Inputs  -  Outputs 

3.4  Data  Characteristics 
ENVIRONMENT 

4.1  Equipment  Environment 

4.2  Support  Software  Environment 
COST  FACTORS 

SYSTEM  DEVELOPMENT  PLAN 

6.1  Introduction 

6.2  Implementation  Requirements 

6.3  Target  Systems 

6.4  Acceptance  Testing 

6.5  Maintenance  Service 

6.6  Software  Maintainability 


2.  Title: 


JOVIAL  J73  Automated  Verification  System  System/ Subsystem 
Specification 

Report  No:  CR-2-947 

Authors:  C.  Gannon,  N.B.  Brooks 

Date:  March  1980 

Abstract: 

This  report  defines  the  software  functions,  database,  and  system 
interface  for  the  J73  Automated  Verification  System.  A  summary  of  the 
system's  requirements  and  capabilities  is  provided  as  background.  The 
software  functions  are  described  according  to  each  independent  processor 
segment.  The  database  is  described  in  terms  of  data  structures,  management  of 
the  database,  and  usage  by  each  processor  segment. 
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3.  Title:  JOVIAL  J73  Automated  Verification  System  Final  Report:  Study 

Phase 

Report  No:  CR-3-947 

Author:  C.  Gannon 

Date:  March  1980 

Abstract : 

This  report  is  primarily  a  review  of  the  state-of-the-art  of  software 
testing  and  verification  with  emphasis  on  techniques  applicable  to  JOVIAL  J73 
programs.  Since  the  project  concerns  the  development  of  computer-based  tool, 
the  need  for  such  a  tool,  the  capabilities  for  the  tool,  and  the  high-level 
design  of  the  tool  are  also  described.  Future  capabilities  for  the  tool  are 
identified . 

This  report  is  also  available  from  the  National  Technical  Information 
Service  (NTIS),  5285  Port  Royal  Road,  Springfield,  Virginia,  22151.  Reference 
RADC-TR-80-26 1 ,  accession  number  A091-190. 
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4.  Title: 


JOVIAL  J73  Automated  Verification  System  User's  Manual 
Report  No.:  CR-4-947/2 
Authors:  C.  Gannon,  R.F.  Else 

Date:  March  1983 

Abstract: 

This  report  describes  the  capabilities  and  user  operation  of  an  Auto¬ 
mated  Verification  System  for  the  JOVIAL  J73  computer  programming  language. 
This  tool,  known  as  J73AVS,  provides  interactive  and  batch  users  with  static 
and  dynamic  program  evaluation,  test  coverage  measurement,  retesting  assis¬ 
tance,  and  automated  reports  that  are  useful  for  debugging,  documentation,  and 
maintenance  of  the  software. 

J73AVS  is  a  command-driven  tool  that  analyzes  one  or  more  JOVIAL  J73 
modules  during  a  session.  J73AVS  maintains  a  database  that  describes  the  J73 
source  being  analyzed.  The  saved  database  allows  incremental  software 
analysis  and  testing.  This  report  describes  tool  operation  during  code 
development,  debugging,  testing,  and  documentation.  Sample  output  is  pro¬ 
vided,  as  well  as  job  control  setups,  resource  estimations,  and  error 
messages . 

J73AVS  operates  on  the  Amdahl  470  (an  IBM  370  equivalent)  at  the  ASD 
Computer  Center  and  the  VAX  11/780  at  ASD/ AD,  both  at  Wright  Patterson  AFB. 
J73AVS  recognizes  both  MIL-STD-1589A  and  -B  versions  of  JOVIAL  J73.  It 
produces  instrumented  code  that  can  be  compiled  with  any  validated  1589A  or  -B 
compiler. 
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5.  Title:  JOVIAL  J73  Automated  Verification  System  Test  Plan 

Report  No.:  CR-5-947 

Authors:  C.  Gannon,  R.F.  Else,  R.K.  Boxer 

Date:  October  1981 

Abstract: 

This  report  provides  a  set  of  test  specifications,  procedures  for 
testing,  and  evaluation  criteria  for  the  formal  testing  of  the  JOVIAL  J73 
Automated  Verification  System  (J73AVS).  J73AVS  will  be  installed  and  formally 
tested  at  ASD/AD,  Wright-Patterson  AFB,  on  the  ITEL  AS/5  and  at  Rome  Air 
Development  Center  (RADC),  Griffiss  AFB,  on  the  DEC20.  Formal  testing  of 
J73AVS  requires  a  validated  JOVIAL  J73  (MIL-STD-1589B)  compiler  and  a  FORTRAN 
compiler  (for  input/output  processing  only). 
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6.  Title: 


JOVIAL  J73  Automated  Verification  System  Program  Maintenance 
Manual  (VAX  and  IBM-Equivalent  Versions) 

Report  No.:  CR-6-947/1 

Authors:  R.F.  Else,  C.  Gannon,  R.K.  Boxer 

Date:  April  1983 

Purpose  of  the  Program  Maintenance  Manual: 

The  Program  Maintenance  Manual  for  an  Automated  Verification  System  for 
JOVIAL  J73,  called  J73AVS,  (PR  No.  B-9-3278)  is  written  to  provide  the 
maintenance  programmer  with  the  information  necessary  to  effectively  maintain 
the  system. 

The  installations  of  J73AVS  are  on  the  Amdahl  470  (IBM  370  equivalent, 
OS/MVS)  at  ASD,  Wright-Patterson  AFB  and  VAX  11/780  computers  at  ASD/ENAF , 
Wright-Patterson  AFB  and  RADC/CO,  Griffiss  AFB.  It  is  intended  that  J73AVS  be 
compatible  with  all  JOVIAL  J73  and  FORTRAN  compilers;  therefore,  the  system  is 
designed  to  be  highly  machine-transf errable  and  not  dependent  upon  particular 
operating  systems  or  other  support  software. 

The  primary  language  in  which  J73AVS  is  written  for  the  IBM  370  equi¬ 
valent  is  JOVIAL  J73  (MIL-Std-1589B).  The  IBM  version  contains  a  few  modules 
written  in  BAL  and  a  few  written  in  structured  FORTRAN.  The  structured 
FORTRAN  processor  source  code  is  included  on  the  IBM  J73AVS  magnetic  tape 
(described  in  Sec.  4.4.1).  For  the  VAX  version  of  J73AVS,  the  language  is 
structured  FORTRAN  (a  subset  of  IFTRAN).  The  structured  FORTRAN  preprocessor 
(written  in  FORTRAN)  will  be  Included  on  that  delivery  tape. 
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JOVIAL  J73  Automated  Verification  System  Program  Specifi¬ 
cation 

Report  No.:  CR-7 -947/1 

Authors:  R.F.  Else,  C.  Gannon 

Date:  September  1983 

Purpose  of  the  Program  Specification: 

The  objective  of  the  Program  Specification  for  an  Automated  Verification 
System  for  JOVIAL  J73,  called  J73AVS,  (PR  No.  B-9-3278)  is  to  describe  the 
program  design  in  sufficient  detail  to  permit  program  production  by  the 
programmer/ coder . 
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8.  Title:  J73AVS  Training  Course 

Report  No.:  CR-8-947 
C.  Gannon 
March  1983 


Author: 

Date: 

Agenda: 


First  Day 


9:00  -  10:30am 
break. 

10:45  -  12:00  noon 
lunch 

2:00  -  3 : 00pm 
break 

3:15  -  4:30pm 


Second  Day 


9:00  -  10:30am 
break 

10:45  -  12:00  noon 
lunch 

2:00  -  3:00pm 
break 

3:15  -  4:30pm 


Third  Day 


9:00  -  noon 


Overview  of  J73AVS  Capabilities 
Selected  Output  Capabilities 
Overview  of  J73AVS  Operation 
J73AVS  as  a  Test  Tool 


Discuss  Class  Exercises 
Review  J73AVS  Analysis  Reports 
Work  on  Sample  Exercises 
Sample  Exercises 


Continued  Workshop 
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6  IMPLICATIONS  FOR  FURTHER  RESEARCH 

There  are  several  fruitful  research  areas  in  which  J73AVS  can  play  an 
important  role.  Some  of  these  areas  call  for  modifying  and  extending  J73AVS 
current  capabilities  and  other  areas  make  use  of  the  tool  as  part  of  an 
overall  system.  The  major  areas  are: 

•  Enhancements  for  testing  embedded  software. 

•  Enhancements  for  improving  software  retesting. 

•  Aids  for  project  management. 

•  Aids  for  software  measurement. 

Each  of  these  areas  is  discussed  in  more  detail  in  the  following  subsections. 

Even  though  JOVIAL  J73  is  a  new  dialect,  its  "phasing  out"  by  the  advent 
of  Ada  was  known  at  the  beginning  of  J73AVS  design  activities.  Therefore, 
further  research  should  support  the  continued  JOVIAL  J73  software  developers 
as  well  as  be  applicable  to  Ada  usage. 

6.1  TESTING  EMBEDDED  SOFTWARE 

Most  of  the  J73AVS  training  course  participants  from  industry  expressed 
one  major  difficulty  in  using  J73AVS  to  test  embedded  software:  the  use  of  a 
mainframe  for  program  execution. 

J73AVS  is  not  intended  to  replace  Performance  Monitoring  Units  (PMUs)  or 
Program  Control  and  Monitoring  Systems  (PCMSs).  These  test  stations  provide 
powerful,  but  low-level  support  to  embedded  computers,  such  as  Mil-Std-1750A. 
As  currently  configured,  typical  PCMS  functions  are  debugging  activities  like 
stepping  through  instructions,  observing  all  input/output  transfers,  main¬ 
taining  "hindsight"  of  the  last  500  instructions,  and  recording  timing 
performance  without  the  overhead  of  added  instructions  to  log  timing. 
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The  type  of  PMU  or  PCMS  developed  by  1750A  computer  manufacturers  might 
consist  of: 


•  A  microprocessor-based  system  with  a  CRT  terminal  for  user 
interface. 

•  A  Mll-Std-1553B  interface  to  pass  information  to  and  from  the 
1750A  machine. 

•  A  RS-232C  interface  for  loading  the  test  program  into  the  1750A. 

•  Peripherals  such  as  magnetic  tape,  floppy  disk,  hard  disk,  and  a 
printer  for  recording  execution  behavior. 

•  Sometimes  connection  to  a  mainframe  computer,  like  a  VAX  11/780, 
for  direct  downlinking  from  the  JOVIAL  J73  compiler  and  VAX  JOVIAL 
linker. 

These  types  of  development  monitors  can  be  more  than  adequate  for 
extensive  software  testing,  as  well  as  the  basic  computer  hardware  and  bus 
checkout,  for  which  they  are  intended.  The  addition  of  J73AVS  in  the  PMU/PCMS 
environment  would  provide  considerable  code  development,  testing,  and  documen¬ 
tation  power  for  embedded  1750A  software  developers  too. 

In  such  an  environment,  J73AVS  would  reside  on  the  VAX.  Static  code 
analysis  and  automated  code  documentation  would  take  place  on  the  VAX,  just  as 
in  non-embedded  software  systems.  Dynamic  testing  (execution  coverage, 
tracing,  timing,  and  assertion  violation  checking)  would  require  these 
modifications  to  J73AVS: 

1.  Rewrite  the  J73AVS  execution-time  data  collection  routine  in  1750A 
assembly  code. 

2.  Replace  the  JTRACE  directive  usage  in  J73AVS  with  calls  to 
appropriate  assembly  output  routines  (the  ! TRACE  directive  is  used 
by  J73AVS  to  report  assertion  violations  and  tracing  results). 

3.  Replace  the  calls  to  the  mainframe's  system  clock  routine  with 
appropriate  1750A  clock  calls,  if  available,  for  the  timing 
analysis. 
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J73AVS  provides  a  great  deal  of  high-level  capabilities  under  simple 
operator  control.  It  Is  recommended  that  embedded  software  developers  take 
advantage  of  these  code  checkout  facilities  before  submitting  1750A  software 
to  final  PMU/PCMS  or  other  low-level  analysis. 

6.2  ENHANCEMENTS  FOR  IMPROVING  SOFTWARE  RETESTING 

J73AVS  currently  offers  these  capabilities  for  retesting  due  to  (1) 
modifying  the  originally  tested  code  and  (2)  wanting  to  Improve  the  thorough¬ 
ness  of  test  data  sets: 

•  Checking  all  module  Interfaces  (l.e.,  COMPOOL  variables  and 
module-level  parameters)  for  changes  and  inconsistencies. 
Ramifications  of  interface  changes  are  checked  throughout  the 
entire  program's  database  (which  Is  built  by  J73AVS  and  saved  on  a 
compressed,  sequential  file). 

•  Data  flow  analysis  (set-use  checking  of  variables)  is  performed  on 
a  designated  (changed)  module,  looking  at  all  interface  variables 
in  that  module's  calling  tree. 

•  In  order  to  improve  thoroughness  of  test  data  sets,  J73AVS 
provides  a  list  of  control  branches  not  yet  executed. 

•  Execution  branch  and  statement  coverage  history  is  saved  on  the 
J73AVS-built  database. 

•  The  set  of  branches  leading  up  to  a  designated  (unexercised) 
branch  is  specified  by  J73AVS. 

J73AVS  provides  a  good  foundation  for  additional  retesting  analysis. 

The  information  preserved  in  the  database,  the  current  assertion  facility, 
interface  checking,  and  data  flow  analysis  can  all  be  advanced  to  provide 
greatly  needed  retesting  assistance. 
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Computer  Sciences  Corporation  ’  (CSC)  recently  reported  some  techniques 
for  improving  software  retesting  in  a  very  narrow,  structural  sense.  The 
basic  approach  that  CSC  took  during  this  study  contract  is  a  good  one: 
identify  a  particular  test  case  with  changed  code.  If  the  changes  do  not 
involve  the  addition  of  branches  or  modules  or  the  implementation  of  new 
functions,  the  test  cases  used  during  the  development  (called  the  testbed  by 
CSC)  may  be  used  to  re-test  the  system.  Most  of  the  time,  however,  new  test 
cases  will  need  to  be  constructed. 

The  reasons  for  making  software  changes  are  generally: 

•  To  correct  development  errors 

•  To  correct  specification  errors 

•  To  satisfy  changing  requirements 

•  To  improve  performance 

Therefore,  changes  to  programs  usually  entail  adding  or  deleting  branches  in 
the  program,  and  they  often  entail  slightly  or  radically  modifying  the 
function  of  the  program. 

To  identify  what  has  changed  in  a  module,  the  re-testing  tool  must  have 
an  extensive  database  containing  sequencing  information,  location  information, 
and  attribute  information  describing  each  module  in  the  software  system. 

Interface  descriptions  of  the  original  and  modified  modules  must  also  be 
maintained  in  the  database.  These  descriptions  would  include  the  entry  point 
names  of  the  module,  a  description  of  the  parameters  of  the  module,  a  descrip¬ 
tion  of  any  global  variables  set  or  used  by  the  module,  the  input/output 
Interfaces  to  the  module,  the  other  modules  called  by  the  module,  and  the 
sequence  of  these  calls.  The  parameters  to  the  module  need  to  be  described 


^Software  Retest  Techniques  Functional  Description,  prepared  for  RADC, 
Computer  Sciences  Corporation,  Feb.  1982. 
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Software  Retest  Techniques  Final  Technical  Report,  prepared  for  RADC, 
Computer  Sciences  Corporation,  Feb.  1982. 


by  their  attributes:  number,  type,  dimensions,  set/use,  ranges  and  enumerated 
values.  Global  variables  should  be  described  by  their  type,  dimension,  use  in 
the  module,  range,  and  initial  value.  Input  and  output  operations  (imple¬ 
mented  as  calls  to  non-JOVIAL  routines  in  J73  programs)  need  to  be  described 
by  their  sequence,  number  of  records  read  or  written,  file  type,  and  record 
format. 

Several  things  must  be  done  in  order  to  re-test  software  effectively: 

•  The  changes  to  the  software  must  be  identified 

•  The  parts  of  the  software  which  need  to  be  re-tested  must  be 
identified 

•  Test  cases  must  be  chosen  to  test  these  parts 

The  major  problem  with  the  CSC  approach  to  software  re-testing  is  that 
it  focuses  almost  entirely  on  a  very  small  class  of  re-testing  needs;  that  is, 
the  class  of  small  changes  to  a  program  in  which  the  actual  structure  is  not 
modified.  In  the  cases  of  adding  or  deleting  branches  to  a  program,  the 
methodology  offers  very  little  assistance.  In  fact,  the  retesting  tool 
specified  in  the  Functional  Description  provides  no  additional  help  over  what 
J73AVS  currently  offers  in  deriving  new  testcases  to  exercise  modified  code. 
The  second  major  problem  is  that  the  methodology  does  not  address  re-testing 
new  functions  or  performance  changes.  To  support  these  testing  activities, 
more  extensive  Interface  checking,  a  formal  use  of  J73AVS  assertions,  and 
implementing  the  execution  performance  instrumentation  component  described  in 
the  J73AVS  Functional  Description.*  Execution  performance  value  ranges  can  be 
used  to  support  unit-level  testing  of  modified  modules. 

6.3  AIDS  FOR  SOFTWARE  MEASUREMENT  AND  PROJECT  MANAGEMENT 

Quantifying  certain  aspects  of  software  development  can  illuminate 
trends  and  problem  areas.  In  particular,  obtaining  software  measurements  can: 


* JOVIAL  J73  Automated  Verification  System  Functional  Description,  C.  Gannon 
and  N.B.  Brooks,  CR-1-947,  General  Research  Corporation,  March  1980. 
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•  Guide  the  testing  process 

•  Record  project  status 

•  Provide  parameters  for  cost  estimation 

•  Quantify  software  quality 

•  Help  formulate  system  testing  requirements 

J73AVS  already  automatically  collects  a  variety  of  measures  through  its 
execution  coverage  analyzer,  static  analyzer,  and  test  history  reporting. 

Other  current  features  such  as  assertion  violation  reporting,  system  overview 
description  (number  and  types  of  modules,  dates  analyzed,  lines  of  code,  etc.) 
and  invocation  structure  reporting  can  be  modified  to  provide  project  status 
monitoring. 

We  recommend  augmenting  the  current  J73AVS  test  history  recording  with 
accumulated  statistics  on  the  number  and  severity  of  static  errors  for  each 
modified  (presumably  "corrected")  version  of  a  module.  As  part  of  measuring 
static  errors,  we  also  suggest  adding  more  extensive  data  flow  analysis 
capabilities  to  J73AVS.  With  JOVIAL  J73*s  rich  data  types  and  scoping  rules, 
many  programming  errors  could  be  uncovered  by  more  exhaustive  analysis  of 
referenced  COMPOOL  variables,  table  or  block  subscripted  values,  etc.  A 
description  of  the  current  data  flow  capabilities  of  J73AVS  is  provided  in  the 
User's  Manual.1 

While  the  programmer  may  be  interested  in  the  types  of  errors  uncovered 
in  each  module,  the  project  manager  may  be  more  interested  in  how  many  modules 
have  undergone  static  analysis,  have  been  executed  with  assertions  turned  on, 
or  have  been  analyzed  for  timing  performance  and  branch  coverage.  Statistics 
of  this  nature  would  be  easy  to  include  in  J73AVS. 


Software  metrics  engineers  usually  lament  about  the  quality  of  software 
error  data  collected  during  (or,  usually,  after)  a  project.  Automatic 


collection  and  output  onto  a  sequential  file  by  J73AVS  would  also  be  an  easy 
and  useful  addition  to  J73AVS.  The  user  could  specify  certain  data  collection 
formats  (such  as  “by  module,"  “by  date  of  analysis,”  “by  error  type,"  etc.) 
for  the  output  file.  Such  a  data  file  could  be  valuable  for  software  problem 
analysis  and  feedback  programs,  error  data  archiving,  partial  input  to  cost 
models,  and  project  management  programs. 


APPENDIX  A 


This  appendix  consists  of  three  technical  papers  that  describe  J73AVS  and 
how  it  was  implemented.  Two  of  the  three  conference  papers  were  presented  to 
audiences  who  have  a  keen  interest  in  high-level  language  support  for  avionics 
applications. 

The  technical  papers  included  in  this  appendix  are: 

1.  A  Debugging,  Testing,  and  Documentation  Tool  for  JOVIAL  J73 

2.  An  Automated  Verification  System  for  JOVIAL  J73 

3.  J73AVS:  A  JOVIAL  J73  Automated  Verification  System 
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A  Debugging,  Testing,  and  Documentation  Tool  for  JOVIAL  J73 


C.  Gannon 


General  Research  Corporation 


Introduction 


This  paper  describes  an  automated  tool 
which  Is  being  developed  to  assist  In  debugging, 
testing,  and  documenting  software  written  In 
JOVIAL  J73.  Many  of  the  techniques  Incorporated 
In  this  tool  have  already  been  found  successful 
In  testing  and  maintaining  FORTRAN  and  JOVIAL  J3 
programs  and  may  be  considered  viable  techniques 
for  future  Ada  tools.  Features  of  this  tool, 
called  J73AVS  (for  JOVIAL  J73  Automated  Veri¬ 
fication  System),  that  distinguish  It  from  ocher 
AVS's  are  primarily  due  to  the  specific  charac¬ 
teristics  of  the  JOVIAL  J73  language  and  the 
applications  programs  written  In  J73.  Addition¬ 
al  distinguishing  features  are  to  provide  a 
realistic  approach  to  testing  program  paths, 
maintain  multi-run  testing  history  Information, 
and  operata  In  both  batch  and  Interactive  modes. 


As  background  material,  characteristics  of 
Che  J73  language  and  applications  programs  are 
described  briefly.  The  paper  then  provides  a 
functional  description  of  the  tool. 


The  Need  For  J73AVS 


The  need  for  this  automated  verification 
system  Is  based  upon  (1)  the  emergence  of  a  new 
JOVIAL  language  dialect  which  will  supersede  the 
previously  approved  JOVIAL  dialects,  (2)  the 
characteristics  of  the  language  that  make  It 
complex  and  error-prone,  (3)  the  type  of  appli¬ 
cations  expected  to  be  written  in  the  language, 
(4)  and  the  standardization  of  certain  testing 


In  an  effort  to  prescribe  a  standard 
policy  for  using  computer  programming  languages 
and  for  testing  computer  programing  language 
compilers,  the  Air  Force  Issued  AF  Regulation 
300-10  In  1976.  Two  JOVIAL  languages,  J3  and 
J73/I,  were  specified  as  Air  Force  standard 
hlgh-order  programing  languages.  Both  JOVIAL 
languages  are  primarily  designed  for  comand  and 
control  system  programing.  They  are  especially 
well  suited  to  large  systems  requiring  efficient 
processing  of  a  large  volume  of  data  with 
complex  structure. 


Another  JOVIAL  language,  J3B,  evolved  from 
J3  for  the  purpose  of  developing  computer 
programs  for  the  Boeing  B-l.  Derivatives  of  J3B 
have  been  widely  used  for  avionics  computer 


programing.  However,  JOVIAL  J3B  Is  not  a 
language  approved  by  AF  Regulation  300-10. 
Therefore,  a  blend  of  J73/I  and  J3B,  plus 
additional  features  not  In  either  language,  has 
been  created  to  satisfy  the  programing  needs  of 
both  the  avionics  and  systems  comunltles.  This 
language,  JOVIAL  J73,  was  specified  In  March 

1979  as  MIL-STD- 1  589a  and  was  refined  in  June 

1980  to  become  MIL -STD-1 589B.  In  the  spring  of 
1980,  AF  Regulation  300-10  was  revised  to  cancel 
both  J3  and  J73/I  languages,  leaving  J73  as  the 
only  approved  JOVIAL  language. 


It  was  the  desire  to  Improve  software 
reliability  that  prompted  the  Air  Force's 
request  for  an  Automated  Verification  System 
(AVS)  to  be  developed  and  made  available  as  soon 
as  possible  following  release  of  validated 
JOVIAL  J73  compilers.  Encouragement  for  an  AVS 
and  other  support  tools  also  came  from  the 
JOVIAL  Users  Group,  a  body  of  Interested 
management  and  technical  people  from  Industry, 
Government,  and  the  Air  Force. 


Characteristics  of  J73  Programs 


As  defined  In  MIL-STD- 1589B,  JOVIAL  J73 
permits  the  Independent  processing  of  functional 
modules  which  comunlcate  through  compools  and 
argument  transmission.  J73  permits  both 
recursive  and  reentrant  procedures  for  effective 
multi-processing.  The  language  provides  a  rich 
variety  of  data  types  and  supporting  data 
manipulation  functions,  making  assembly  code 
programing  unnecessary  for  most  applications. 
However,  except  for  a  trace  directive  which 
supplies  limited  output  facility,  there  is  no 
Input/output  capability  In  the  language. 
Linkages  to  assembly  or  alternate-language 
routines  are  required  for  Input  and  output. 


Storage  allocation  for  daca  objects  can  be 
both  automatic  (in  which  storage  Is  released 
when  control  exits  from  the  program  unit)  or 
static  (in  which  storage  space  Is  saved  through¬ 
out  the  entire  execution  of  the  program). 
Automatic  allocation  uses  storage  efficiently 
but  makes  certain  data-usage  errors  possible. 


The  DEFINE  construct  associates  a  name 
with  a  text  string  such  that  whenever  that  name 
Is  referenced,  the  text  string  replaces  It. 
DEFINE  statements  can  be  nested  and  can  be 
redefined  based  upon  scope.  Thus,  while  the  ca- 
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pablllty  Is  extremely  useful,  It  adds  another 
dimension  of  complexity  to  JOVIAL  programs. 


Unfortunately  for  advocates  of  structured 
programming,  the  control  statements  In  JOVIAL 
J73  are  not  confined  to  the  'structured  program¬ 
ming"  constructs  of  sequential  flow,  IF-THEN- 
ELSE,  and  WHILE-loops.  The  language  does  at 
least  have  these  constructs,  so  that  programmers 
can  write  structured  code  If  they  desire. 
However,  unstructured  statements  as  GOTO, 
FALLTHRU,  EXIT,  and  ABORT  are  also  permitted. 
The  GOTO  statement  allows  transfer  from  the 
outside  of  an  IF  or  CASE  construct  Into  the  body 
of  the  IF  or  CASE.  GOTO  statements  can  also  be 
directed  to  labels  that  are  external  to  a 
program  unit,  If  the  label  is  passed  as  a 
parameter.  The  FALLTHRU  statement  allows 
control  to  pass  from  one  CASE  alternative  to 
another  without  making  the  test  normally 
required  at  each  CASE  option.  The  EXIT  state¬ 
ment  allows  escape  out  of  an  Immediately 
enclosing  loop.  The  ABORT  statement  provides 
transfer  of  control  to  the  label  specified  In 
the  most  recently  executed,  currently  active 
procedure  having  an  ABORT  phrase .  Thus ,  control 
transfer  Is  not  defined  until  execution  time. 


The  unstructured  control  statements 
provide  flexibility  and  execution-time  efficien¬ 
cy;  but  at  the  same  time  they  Increase  the 
chance  of  conlttlng  errors  and  make  the  program 
more  difficult  to  understand.  Since  60Z  of  the 
total  cost  of  software  Is  generally  attributed 
to  maintenance,  source  code  acrutablllty  Is 
Important. 


J73AVS  will  provide  extensive  static  and 
data-flow  analysis  to  detect  and  report  possible 
errors  regarding  control  transfers,  data 
contention  due  to  static  allocation,  uninitial¬ 
ized  variables,  structurally  unreachable  code, 
potential  Infinite  loops,  etc.  Program  analysis 
reports  can  be  generated  on  command  by  the  user 
to  describe  such  detailed  Information  as  DEFINE 
usage,  label  references,  symbol  properties,  and 
global  data.  Scatlc  and  daca-flow  analysis  are 
testing  and  program  analysis  techniques  In  which 
che  program  la  not  executed  with  real  data. 
Rather,  the  code  Is  analyzed  for  inconsistencies 
(static  analysis)  or  for  variable  set-use 
behavior  (data  flow  analysis)  using  symbolic 
execution  through  a  dlrected-graph  of  the 
program. 


Characteristics  of  Application  Programs 


The  programs  that  will  be  Implemented  In 
JOVIAL  J73  will  be  of  similar  nature  to  those 
written  In  the  separate  JOVIAL  dialects:  J3, 
J3B,  and  J73.  Applications  will  be  for  navi¬ 
gation,  Information  management,  flight  controls, 
communications,  etc.  The  software  character¬ 
istics  of  the  applications  are  varied.  For 
example,  fllghc  control  software  has  the 
following  characteristics: 


synchronization 
distributed  processing 


structurally  simple  control  state¬ 
ments 


simple  data  types 
real-time  processing 


On  the  other  hand,  applications  such  as  command 
and  control  systems  have  very  different  charac¬ 
teristics  such  as: 


batch  and  Interactive  modes 
complex  data  structures 
complex  control  structures 
large,  monolithic  modules 
non-real-time  processing 


Avionics  applications  are  often  destined 
for  small  on-board  computers.  For  those 
computers  not  having  a  JOVIAL  J73  or  J73-subset 
compiler,  the  programs  are  developed  on  a  host 
machine  and  cross-compiled  to  the  target 
machine.  There  are  no  software  checkout  tools 
available  on  these  small  computers,  so  an  AVS 
operating  on  the  host  computer  must  supply  as 
much  assistance  as  possible  to  detect  errors  In 
program  performance  and  assure  some  level  of 
testing  thoroughness  before  the  program  Is 
cross-compiled. 


Command  and  control  systems,  on  the  other 
hand,  tend  to  be  very  large  (several  hundred 
thousand  lines  of  code).  They  also  tend  to 
evolve  as  needs  change.  Therefore,  not  only  is 
testing  a  major  problem  —  code  modification  and 
retesting  only  what  Is  necessary  are  also 
difficult  tasks.  In  the  face  of  these  problems, 
one  of  the  most  valuable  assets  of  any  software 
support  tool  Is  the  ability  to  automatically 
produce  concise  but  helpful  program  documen¬ 
tation. 


Functional  Description  of  J73AVS 


This  section  presents  a  brief  description 
of  the  capabilities  of  J73AVS  and  describes  In 
what  phases  of  the  software  life  cycle  che 
capabilities  should  be  used.  A  thorough 
description  Is  provided  In  the  Functional 

Description. 


Our  approach  to  the  design  of  an  AVS  for 
JOVIAL  J73  Is  to  provide  automated  assistance 
for 

-  program  development 

-  debugging 
testing 
retesting 


The  approach  excludes 

-  verification  of  requirements 
verification  of  specifications 
automated  design  aids 

-  formal  program  verification  (proof 
of  correctness) 
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The  techniques  for  automating  these  processes 
are  not  developed  vail  enough  to  be  reliable  for 
general-purpose,  large  software  systems. 


The  specifications  for  the  J73  dialect  and 
compilers  Include  rigorous  data-type  checking 
and  scope  rules.  Tha  language  allows,  hovevsr, 
constructs  and  control  structures  which  demand 
caution  In  their  usage  (such  as  recursive  and 
reentrant  procedures,  juape  Into  certain  control 
structures,  abnormal  exits,  etc.).  Further,  the 
language  does  not  contain  a  mechanism  for 
specifying  expected  behavior  or  reporting 
user-specified  abnormalities  (since  there  is  no 
Input/output  facility). 


J73AVS  will  not  duplicate  the  static 
consistency  checking  of  the  coapller,  but, 
rather,  provide  the  following  set  of  facilities 
to  support  program  development,  debugging, 
testing,  maintenance,  and  documentation  of 
JOVIAL  J73  programs: 


1. 

2. 

3. 


Logical  assertions  and  timing  probes 
Static  and  data  flow  analysis 


Program  structure  and  characteristic 
reporting 


4. 

5. 

6. 


Statement 

analysis 


performance  dynamic 


Branch,  path,  and  program  unit 
execution  coverage  enalysls 


Branch  and  program  unit  execution 
trace  analysis 


7. 

8. 
9. 


Execution  timing  analysis 
Structural  retesting  assistance 
Test  history  reporting 


J73AVS  will  support  interactive  and  batch 
facilities  since  the  various  stages  of  program 
development  through  teetlng  and  maintenance  lend 
themselves  to  both  modes  of  operation.  The 
command  language  will  be  slmller  for  Interactive 
and  batch  usage,  except  that  the  Interactive 
user  will  be  prompted  for  Information  where 
necessary. 


Summary  of  Capabilities 


A  sumary  of  capabilities  Is  provided  as  a 
flow  diagram  In  Fig.  1.  This  diagram  describes 
the  primary  functions  supported  by  J73AVS  as 
well  as  che  sequence  In  which  they  are  per¬ 
formed.  Figure  2  shows  the  Interaction  between 
J73AVS  and  the  user.  The  user  can  direct  the 
sequence  of  analysis  activities,  using  Infor¬ 
mation  provided  at  each  stage  of  processing. 


Although  J73AVS  will  exist  as  a  single 
program  (with  overlays  to  keep  it  compact),  It 
la  best  considered  as  a  collection  of  toola  or 
facllltlea  with  which  the  user  Interacts.  Some 
of  che  facilities,  such  as  automated  documen¬ 
tation,  static  error  reporting,  and  Instrum¬ 
entation,  are  completely  automated  and  require 
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only  that  the  user  initiate  the  tasks  by 
coaund.  Other  processes,  such  as  execution- 
time  data  collection  or  retesting  assistance, 
require  more  Information  from  the  uaer  like  test 
data  Input  and  teat  target  selection. 


J73AVS  provides  detailed  Information  both 
statically  and  dynamically  about  the  program 
being  analyzed.  It  is  the  role  of  the  user  to 
direct  the  processing  performed  by  J73AVS,  to 
analyze  the  output  produced  by  J73AVS,  and  to 
determine  subsequent  action. 


The  role  of  J73AVS  In  the  software 
development  cycle  Is  to  provide  automated 
aasiatance  wherever  possible  during  the  program 
development  and  maintenance,  debugging,  teetlng, 
and  retesting  phases  of  the  cycle.  Because 
J73AVS  systematically  provides  execution 
coverage,  timing  analysis,  execution  performance 
reporting,  and  test  history,  It  la  expected  to 
be  valuable  to  Independent  verification  and 
validation  contractors  as  well  as  software 
developers  of  J73  programs. 


The  user  of  J73AVS  plays  an  active  part  In 
the  cycle  as  shown  In  Fig.  3.  This  figure 
partitions  the  phases  of  the  development  cycle 
and  shows  the  flow  between  the  automated 
processing  of  J73AVS  and  user-supplied  Input  or 
direction. 


Using  Fig.  3  as  a  basis,  a  typical 
sequence  of  J73AVS-supported  processing  can  be 
described  as  follows: 


1. 


JOVIAL  J73  source  text  Is  generated 
and  provided  to  J73AVS  as  one  or 
more  compilable  modules. 

J73AVS  produces  program  analysis 
reports  showing  control  structure, 
symbol  usage,  calling  hierarchy, 
etc.,  as  well  as  a  static  analysis 
report  showing  errors  and  dangerous 
programming  practices. 


3. 


4. 


Using  the  reports  as  s  guide,  the 
source  modules  can  be  modified  or 
new  modules  sdded  to  the  program. 

J73AVS  Identifies  the  Interaction  of 
the  new  or  modified  modules  with  the 
rest  of  the  program;  thla  Informa¬ 
tion,  In  turn.  Is  used  as  the  basis 
for  modifying  other  modules. 


5. 


6. 


For  dynamic  debugging,  the  program 
Is  lnstrumsnted  (l.e.,  calls  to  data 
collection  routines  are  automatical¬ 
ly  Inserted)  by  J73AVS  and  executed 
with  an  Initial  test  case  supplied 
by  the  user . 

J73AVS  reports  assertion  vlolstlons, 
If  sny,  and  generates  an  evaluation 
of  statement  and  variable  perform¬ 
ance. 


Using  this  evaluation ,  the  uaer  may 
chooae  to  generate  additional  teat 
data  to  pinpoint  errore  or  inetru- 
aent  other  aodulee  for  additional 
dynamic  debugging. 


Like  debugging,  the  aaae  procedurea 
of  teat  data  generation,  lnstruaen- 
tation,  and  execution  are  performed 
for  teetlng  but  for  a  different 
goal:  rather  than  detecting  and 
locating  errora,  teetlng  aiaa  to 
deaonatrate  the  abeence  of  errora 
and  chat  the  software  conforms  to 
Its  specification.  Therefore, 
J73AVS  produces  execution  analysis 
reports  In  terns  of  the  thoroughness 
of  execution  coverage  and  violations 
of  asserted  behavior. 


The  user  evaluates  execution 
coverage  and  other  prograa  perform¬ 
ance  output,  along  with  the  pro- 
graa's  own  execution  results  and  the 
prograa  specification,  to  determine 
if  testing  la  complete. 


J73AVS  provides  branch  sequence 
Information  to  retest  targets  chosen 
by  the  uaer.  A  test  history  of 
execution  coverage  and  assertion 
violations  assists  the  user  In 
choosing  targets  for  retesting. 


SOFTWARE  LIFE-CYCLE  PHASES 


PROGRAM  DEVELOPMENT/ 
MAINTENANCE 


DEBUGGING 


GENERATE 

REPORTS 


INSTRUMENT 


Figure  3. 


IDENTIFY  OBTAIN 

PROGRAM  PERFORMANCE  INSTRUMENT 

INTERACTION  ANO  ASSERTION 

-  INSULTS  - 

Role  of  J73AVS  tn  the  Software  Development  Cycle 


PROVIDE 

RETESTING 

ASSISTANCE 


Program  Development  and  Maintenance 


Executable  assertions  permit  a  prograaaar 
to  specify  expected  behavior.  J73AVS  supports 
the  technique  of  embedding  programer-speclfled 
assertions  Into  the  code  through  the  use  of  the 
ASSERT  keyword  followed  by  any  legal  logical 
(Boolean)  expression.  Logical  assertions  csn  be 
used  for  executlon-tlae  exception  reporting, 
stress  testing,  test  data  generation  filtering, 
and  (left  as  coaments  In  the  source  code) 
stating  ln-llne  specifications. 


properties  of  all  or  specified 

symbols 


declaration  and  reference  of  labels 

(statement  names) 


declaration  and  reference 

user-defined  data  types 
declaration  and  reference 


constants 


usage  of  external  reference  (REF) 
and  definition  (DEF) 


To  assist  with  reliable  system  develop- 
aent,  maintenance,  and  docuaentatlon ,  J73AVS 
will  provide  substantial  prograa  analysis  re¬ 
porting  on  structural  hierarchy,  syabol  usage , 
invocations,  certain  J73  constructs,  and  systea 
characteristics.  The  user  has  control  over 
obtaining  high-  or  low-level  Information  through 
che  cosmand  language.  The  types  of  prograa 
analysis  reporting  Include  the  following: 


declaration  and  reference  of  DEFINE 
text  strings 

description  of  prograa  units  on  the 

database 


Debugging 


Indented  source  listing  with  control 
structure  Identification 


syabol  cross  reference  with  set-use 
information 


coapool  syabol  description 


Noraal  compilation  using  JOVIAL  J73 
compilers  will  detect  many  syntax  and  semantic 
errors.  Additional  errors  such  as  uninitialized 
variables,  possible  infinite  loops,  unreachable 
code,  certain  Improper  constructs,  and  dangerous 
coding  practices  (like  transferring  into  CASE  or 
IF  statements)  will  be  reported  by  J73AVS.  The 
user  can  comaund  different  levels  of  static 
reporting. 
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Dynamic  debugging  will  be  eupporced  by 
stataaant  execution  performance  and  aaaartloo 
exception  reporting.  Stateaent  execution 
perforaance  providea  execution  counta  of 
etateaenta,  value a  and  rangea  of  varlablea  In 
aaalgnaenta  and  loops,  and  the  execution 
behavior  of  IF  etateaenta.  Thla  debugging 
lnforaatlon  appeara  adjacent  to  the  aource 
etateaenta  theneelvee.  which  aaalata  the  teak  of 
code  correction.  The  execution  of  tiaing  probe* 
(lneerted  by  command)  can  be  reported  In  the 
debugging  perforaance  report  at  the  user* a 
requeat . 

When  the  prograa'a  execution  behavior 
deviates  froa  the  acceptable  logical  behavior 
specified  by  the  embedded  assertions.  It  will  ba 
reported  during  execution.  This  la  called  an 
assertion  violation.  The  user-supplied  asser¬ 
tions  reaaln  relatively  transparent  to  the 
prograa  until  they  are  violated;  at  that  tlaa 
the  violation  la  reported  along  with  the  module 
name  and  source  statement  number  where  the 
violation  occurred. 

Testing 

When  used  In  conjunction  with  static 
checking  and  statement-level  performance 
analysis,  structure-based  testing  can  uncover 
errora  due  to  untested  branches  (where  a  branch 
la  a  control  flow  outcome  due  to  a  decision 
statement)  or  Improper  sequences  of  branchsa. 
J73AVS  will  provide  execution  tracing  of  prograa 
units  and  branches  and  execution  coverage 
analysis  of  prograa  units,  branches,  and 
sequences  of  branches  (paths).  Further,  J73AVS 
will  assemble  the  timing  lnforaatlon  froa 
prograa  unit  tracing  and  uaer-supplled  tlalng 
probes  Into  an  execution  tlalng  report. 

Although  an  AVS  can  provide  an  objective 
aeasure  of  testing  thoroughness  In  terns  of 
stateaent  or  branch  execution  coverage,  fre¬ 
quently  errora  In  software  arm  overlooked  during 
testing  because  only  certain  sequences  of 
branches  are  ever  executed.  Obviously,  It  la 
generally  laposslble  to  define  all  paths  In 
prograas  because  of  loops.  Furthsraore,  the 
most  likely  subset  of  paths  to  test  can  bsst  bo 
Identified  by  a  person  faailiar  with  the 
function  of  the  prograa.  The  most  efficient 
role  of  an  AVS  In  thla  regard  la  to  Identify  the 
set  of  control  paths  between  two  stateaent*  In  a 
prograa  unit  (an  lnvokabls  unit  of  code)  to 
which  the  human  tester  attaches  Importance.  Of 
the  aet  of  patha  Identified  by  the  tool,  the 
user  can  choose  those  that  are  to  be  analysed 
for  coverage  during  execution.  If  the  set  of 
paths  la  too  large  to  enuaerate,  a  descriptive 
aeaaage  will  be  Issued  and  the  user  allowed  to 
choose  another  pair  of  stateaent*  for  path 
Identification. 

Retesting  Assistance 

Retesting  software  is  performed  when 
analysis  shows  that  prior  testing  Is  inadequate 
(Insufficient  branch  coverage,  not  all  functions 


demonstrated,  etc.)  or  when  prograa  changes  have 
taken  place.  The  proper  approach  to  take  In 
retesting  la  highly  dependent  upon  the  charac¬ 
teristics  of  the  prograa  being  teeted  aa  well  as 
the  aaasures  being  used  to  evaluate  testing 
coaplateneaa. 

In  ordar  to  determine  the  sequences  of 
branches  which  aust  be  executed  to  reach  an 
untested  branch  or  stateaent,  the  user  can 
request  that  the  "reaching  set"  be  computed 
between  two  specified  etateaenta  (or  froa  the 
prograa  unit's  entry).  The  user  can  also 
raqueat  a  list.  In  terms  of  branches,  of  all 
control  paths  between  two  specified  statements. 
If  certain  loop  structures  aake  this  list  lapos¬ 
slble,  subsets  of  the  paths  will  be  Identified. 

With  the  control  flows  Identified,  the 
user  can  develop  new  test  data  by  backtracking 
through  the  prograa  to  the  Input  space,  using 
statement  execution  perforaance  reports,  module 
Interaction  and  Invocation  reports,  and  execu¬ 
tion  coverage  Information  for  each  teatcase. 
Unfortunately,  automatic  test  data  generators 
which  uaa  symbolic  execution  are  not  yet 
developed  to  the  point  of  being  general-purpose, 
aaay  to  uaa,  or  reliable. 

The  cumulative  teat  coverage  history 
aaintalnad  by  J73AVS  will  be  useful  In  attaining 
tasting  goals  and  determining  targets  for 
retesting.  Typically,  the  results  of  executing 
test  cases  ars  difficult  to  manage.  Prograa 
unit  and  branch  coverage  lnforaatlon  will  be 
saved  In  a  concise  way  on  the  database  for  each 
Cast  case.  The  results  of  subsequent  execution 
runs  can  be  added,  providing  a  cumulative  report 
of  all  tests.  Also  saved  in  the  history 
database  table  will  be  any  assertion  violations 
that  occur.  This  will  provide  a  mechanism  for 
Identifying  which  input  test  case  caused  a 
violation. 
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ABSTRACT 


The  introduction  of  Che  J73  dialect  of  che 
JOVIAL  progressing  language  created  a  need  for  new 
software  tools  to  help  develop,  teat,  and  maintain 
J73  software  systems.  This  need  la  especially 
great  in  software  for  aviation  electronics 
(avionics),  where  rigid  functional  and  reliability 
requirements  sake  the  autoaation  of  software 
analysis  a  necessity. 


language's  complexity  and  Its  varying  applications 
reinforce  the  need  for  an  AVS  to  help  analysts  and 
prograwara  handle  the  difficult  task  of  develop¬ 
ing  and  maintaining  J73  software  aystsaa. 


To  fill  this  need.  General  Research 
Corporation  (CSC)  has  developed  the  JOVIAL/J73 

Automated  Verification  System  (J73AVS) . * ’ ^  It  la 
a  tool  that  proceasas  J73  source  coda  Ilka  a 
compiler,  but  maintains  a  permanent  database  of 
Information  making  repeated  processing  of  source 
code  unnecessary.  It  operates  In  either  batch  or 
interactive  mode.  This  paper  describes  the 
features  and  use  of  J73AVS. 


J73AVS  PUNCTIOWS 

J73AVS  was  specifically  designed  to  be  used 
In  the  post-design  phases  of  the  software  life 
cycle:  program  development,  debugging,  tasting 
(and  retesting) ,  and  maintenance.  Since  automated 
tools  already  exist  for  the  specification  and 
verification  of  requirements  and  design,  we  felt 
that  concentrating  on  the  actual  coda  production 
and  maintenance  phases  would  result  in  a  more 
powerful,  concentrated  set  of  capabilities. 
J73AVS  can  also  produce  a  wide  variety  of  progrsn 
documentation;  although  this  activity  la  not 
generally  Included  In  the  software  life  cycle.  It 
is  often  a  weak  link  In  the  chain  holding  together 
am  otherwise  strong  system. 


BACKGROUND 

In  1976,  In  an  effort  to  standardise 
computer  programming  languages  and  compilers,  the 
Air  Force  Issued  AF  Regulation  300-10  which 
specified  two  JOVIAL  languages,  J3  and  J73/I,  as 
standards  for  avionics  and  ciimiiii  lest  Ion  systems. 
Another  JOVIAL  dialect,  J3B,  had  evolved  from  J3 
primarily  for  use  In  the  Boeing  B-l  computer 
system,  although  J3B  was  not  approved  by  the  AF 
regulation.  A  new  dialect,  J73,  specified  in  1979 
as  MIL-STD-1589A  was  a  blend  of  J73/I  and  J3B, 
plus  some  new  features;  this  specification  was 
refined  in  aid-1980  to  become  MIL-STD-1589B.  In 
the  spring  of  1980,  AF  Regulation  300-10  was 
revised  to  eliminate  J3  and  J73/I,  leaving  J73  aa 
the  only  approved  JOVIAL  dialect. 


The  tool's  capabilities  were  designed  to 
complmeent  those  of  the  J73  compiler,  so  that  the 
two  together  would  cover  aa  many  trouble  spots  as 
possible.  The  compiler  provides  rigorous 
data-type  checking  and  scoping  rules ,  but  there 
are  cases  where  the  prograsmar  can  circumvent 
these  checks,  idiether  intentionally  or  not.  In 
addition,  the  language  Itself  allowe  potentially 
disastrous  control  constructs  such  ss  jumps  Into 
CASE  and  IF  statements,  abnormal  axlts  from 
procedures,  stc.,  aid  It  la  beyond  the  scope  of 
the  compiler  to  Issue  warning  messages  for 
programs  which  use  them.  He  also  wanted  to  avoid 
duplicating  capabilities  of  other  tools  which  have 
a  bearing  on  J73;  SDDL  (System  Design  snd 
Documentation  Language),  for  example,  works  at  the 
pseudo-code  level,  and  the  Jovial  Interactive 


The  request  for  a  J73  Automated  Verification 
System  (AVS)  was  part  of  the  Air  Force's  dsslre  to 
improve  software  reliability,  and  It  was  planned 
for  release  aa  soon  as  possible  after  the  release 
of  new  validated  J73  compilers.  Encouragement  for 
an  AVS  and  ocher  support  tools  cams  also  from  ths 
JOVIAL  User's  Group,  a  body  of  Interested 
oumagement  and  technical  people  from  industry. 
Government,  and  the  Air  Force. 


Debugger  will  operate  at  execution  time. 


J73AVS  provides  ths  following  set  of 
functions: 


1.  Static  mid  data-flow  analysis 

2.  Program  structure  and  Interface  report¬ 
ing 


JOVIAL  J73  is  an  extremely  rich  and  complex 
language,  and  will  be  used  for  a  fairly  wide 
variety  of  applications,  including  navigation, 
information  aanagemenc,  flight  controla,  communi¬ 
cations,  and  co am  and  and  control  systems.  The 


3.  Execution  coverage  analysis  (at  state¬ 
ment,  branch,  and  procedure  levels) 


4.  Exeuctloo  tracing  analysis  (at  branch. 


procedure,  and  variable  levels) 
3.  Execution  timing  analysis 
6.  Test  history  reporting 
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7.  Structural  retesting  assistance 

8.  Logical  aaaertlona  for  reporting  devia¬ 
tion  from  expected  prograa  behavior 

9.  A  wide  range  of  docuaentation  report* 

A  summary  of  all  the  J73AVS  capabilities  la 
provided  in  Figure  1.  This  dlagraa  describes  the 
primary  functions  supported,  as  well  as  the 
sequence  In  which  they  are  performed .  Figure  2 
shows  the  Interaction  between  J73AVS  and  the  user; 
a  coaaand  language  Is  used  to  direct  the  sequence 
of  actlvltlee.  The  coaaand  language  Is  identical 
In  either  batch  or  Interactive  aode.  It  1* 
keyword-oriented,  has  parameters  that  specify 
options,  allows  abbreviations,  and  la  flexible  In 
the  order  of  parameters.  In  interactive  aode, 
proapts  and  a  help  function  assist  users. 

Although  J73AVS  exists  as  a  single  prograa, 
It  la  really  a  collection  of  toola  or  facilities 
froa  which  the  user  can  choose.  Some,  such  as 
aucoaated  docuaentation,  static  arror  reporting, 
and  Instrumentation,  are  completely  automated  and 
require  only  a  single  cnMsnd  froa  the  user. 
Others,  such  as  executlon-tlae  dsta  collection  and 
retesting  assistance,  require  aore  input  and 
Interaction  with  the  user. 

SAMPLE  OUTPUT 

Some  saaplee  of  actual  output  give  a  quick 
picture  of  how  J73AVS  can  be  used.  Figure  3(a)  is 
a  J73AVS  report  summarizing  the  contents  of  the 
user's  database,  itealxed  by  module  neae.  It 
shows  each  module' e  naae,  type  (compool ,  prograa, 
or  procedure),  number  of  statements,  prograa  units 
(individual  lnvokable  procedures),  coapool 
references,  DEFINE  naae  declarations,  and  creation 
and  update  dates.  J73AVS  databases  are  retained 
froa  run  to  run,  allowing  then  to  grow  incremen¬ 
tally  as  a  system  Is  being  developed.  One 
database  may  be  uaed  to  retain  data  about  an 
entire  system;  alternatively,  different  program¬ 
mers  may  maintain  different  databases  which  can  be 
combined  at  a  convenient  point  In  development. 
Figure  3(b)  Is  another  fora  of  J73ATS  database 
euaaary,  Itemized  by  progrea  unit  naae.  Each 
procedure’s  parent  (enclosing  procedure,  If  any) 
Is  given,  along  with  nesting  Information. 

The  next  report,  Figure  4,  shows  execution 
timing  data  produced  by  J73AVS.  This  report  Is 
obtained  by  Ins truaen ting  one  or  aore  procedures 
with  tlalng  probes  (Inserted  automatically  on 
comm  and  by  J73AVS),  executing  the  lnstmented 
code,  then  using  J73AVS  to  analyze  the  'audit* 
flic  produced  by  Che  execution.  The  bottom  half 
of  this  exaaple  shows  that  two  segments  of  cod* 
were  chosen  by  the  J73AVS  user  for  monitoring 
execution  time.  These  "clock  Intervals'  need  not 
be  wholly  contained  In  the  saae  procedure  or 
module.  The  execution  times  shown  are  In 
milliseconds  and  are  a*  accurate  ae  the  system- 
level  CPU  clock. 

The  user  may  also  Instrument  source  code  to 
record  which  control  branches  are  covered 
(executed)  during  a  test  execution  of  the  code. 
J73AVS  reports  exactly  which  branches  have  or  have 


not  been  hit,  and  maintains  a  history  of  the 
coverage  results  In  Its  database.  Statement  and 
branch  execution  coverage  can  be  reported  by 
J73AVS  as  part  of  a  source  listing,  or  more 
abbreviated  reporta  can  be  selected  at  user 
option.  For  exaaple.  Figure  5  ahows  the  "NOT  HIT" 
report  for  two  testcases  (aectlons  of  source  code 
chosen  as  test  boundaries  by  the  user)  and  a 
cumulative  branch  coverage  history.  The  branch 
nuabers  in  this  report  can  be  keyed  back  to  the 
source  cod*  through  several  other  J73AVS  reports. 

A  system-level  branch  coverage  report  Is 
available  and  Is  shown  in  Figure  6.  This  report 
shows  the  results  of  all  testing  so  far  and 
Includes  the  nuaber  or  procedure  Invocations, 
branches,  and  stateaents  hit,  and  percentage  of 
total  coverage. 

THE  HOST/TARGET  DICHOTOMY 

A  major  concern  In  avionics  software  Is  the 
difficulty  of  performing  thorough  dynaalc  testing 
on  embedded  software.  Target  computers  are 
usually  much  saaller  than  the  original  host,  with 
limited  memory  and  processing  resources  and  little 
or  no  I/O  capability.  Dynamic  testing,  and 
measuring  Its  thoroughness,  are  difficult  problems 
In  these  small  real-tlae  environments. 

The  lnstruaentatlon  capabilities  of  J73AVS 
offer  a  solution  to  this  problem.  The  software  to 
be  tested  can  be  Instrumented  autor  ' ically  under 
the  user’s  control,  and  the  insulting  code 
compiled,  linked,  and  loaded  (but  not  executed)  on 
the  host  computer,  then  transferred  to  the  target 
on  magnetic  tape,  punched  cards,  or  whatever  I/O 
medium  Is  available.  Executing  the  instrumented 
code  on  the  target  produces  an  audit  file  for 
subsequent  analysis  by  J73AVS  on  the  host. 
Alternately,  If  the  target  has  enough  main  memory, 
or  lacks  an  output  device  for  the  audit  file,  the 
J73AVS  data  collection  and  analysis  routines  can 
easily  be  modified  to  use  a  "HITS'  array  (one  cell 
per  branch,  each  holding  a  "hits'  counter); 
post-execution  analysis  of  this  data  would  then 
take  place  on  the  target  computer. 

Since  the  user  has  complete  control  over 
which  segments  of  code  to  Instrument,  overhead  and 
code  expansion  can  be  minimized;  testing  can  be 
done  In  segments,  or  the  testers  can  "zero  In"  on 
a  particularly  critical  procedure  or  module.  This 
Is  a  significant  gain  over  previous  approaches 
which  Involve  either  the  tedium  of  Inserting  the 
software  probes  by  hand,  or  a  lack  of  a  fine¬ 
grained  automatic  control  over  the  segments  of 
code  to  be  Instrumented,  thus  causing  unnecessary 
overhead. 

This  method  of  testing  on  a  small  target 
computer  has  already  been  proven  with  a  tool  named 

EAVS  (Extensible  AVS),  also  developed  at  GRC.4 
EAVS  was  an  early  precursor  of  J73AVS,  and  the 
software  technology  advances  since  that  time  make 
J73AVS  a  very  Important  testing  tool  for  embedded 

systems. 
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LOGICAL  ASSERTIONS 

This  is  a  slap  la  yat  po warful  technique  in 
which  tha  progriatr  lnsarts  apaclal  stataaants 
callad  'assertions'  into  tha  source  coda  that 
specify  the  expected  behavior  of  the  prograa  at  a 
given  point.  This  facility  provides  tha  aoet 
payoff  when  assartlons  ara  progressed  during 
design  or  early  coding  stages.  For  example,  tha 
assertion 

*.  ASSERT  (  STACK’ POINTER  >-  0  )  * 

states  that  tha  value  of  tha  data  Its*  STACK’ 
POINTER  la  at  least  taro. 

Assertions  provide  a  vary  convenient  way  of 
reporting  ex ecu C ion- tine  deviations  froa  expected 
or  required  behavior  and  can  be  used  for  stress 
testing,  test-data  generation,  etc.  After  code  Is 
thoroughly  checked  out,  they  can  slaply  be  left  as 
c omenta  in  the  code  (as  shown  above).  One  of  the 
lnstruaentatlon  options  provided  by  J73AVS  expands 
these  atatementa  Into  executable  code;  when  the 
expanded  code  la  executed,  violations  of  the 
asserted  conditions  are  reported  as  part  of  the 
prograa  output.  For  exaaple,  a  negative  value  of 
STACK’POINTER  (above)  would  produce  a  aesaage  like 

IN  MODULE  POP ’STACK:  ASSERT  FALSE  AT  STMT  149 

A  special  FAIL  stateaent  la  also  provided 
for  the  programer  to  use  In  specifying  'contin¬ 
gency  code"  for  deviations  froa  the  asserted 
conditions.  In  developing  J73AVS,  we  made  liberal 
use  of  assertions  to  help  us  debug  and  test  the 


coda,  aid  the  assertions  still  exist  In  the  source 
code  of  tha  tool. 

OTHER  FUNCTIONS  OF  J73AVS 

Besides  those  discussed  above,  J73AVS 
provides  tha  following: 

e  indented  source  listings 

e  single-  or  aultl-aodule  syabol  cross 

references  with  set-use  lnforaatlon. 
The  J73AVS  user  can  select  syabol  naaes 
and/or  data  types. 

e  static  analysis  to  pinpoint  error-prone 
coding  practices,  alsaatched  formal  and 
actual  paraaeters  (when  the  compiler  Is 
unable  to  do  so),  unused  labels,  data 
contention  In  recursive  or  reentrant 
procedures,  possible  Infinite  loops,  and 
unreachable  code. 

s  Instrumentation  to  provide  stateaent  or 
branch  execution  counts  and  procedure, 
branch,  or  user-specified  variable 
tracing. 

e  DEFINE  name  declarations  and  usage 
cross-ref  er ence . 

e  module  and  system  Interface  descrip¬ 
tions,  including  calling  trees,  etc. 

SUMMARY 

J73AVS  Is  a  tool  that  can  lower  software 
production  costs  and  Improve  software  reliability. 
It  provides  a  framework  for  the  use  of  standard- 


lzed ,  organized  testing  measures,  and  is  useful  as 
a  software  development  and  maintenance  tool. 


It  can  operate  in  either  batch  or  interac¬ 
tive  mode  and  uses  a  flexible  command  language 
complete  with  a  "HELP"  feature.  Keeping  J73AVS  as 
machine- independent  as  possible  has  been  a  goal 
throughout  its  development.  Dynamic  testing  on 
target  computers  can  be  performed  by  analyzing  the 
execution-data  file ,  produced  by  any  target  that 
can  output  sequential  files  to  off-line  storage, 
on  the  host  computer.  Current  and  planned  host 
computers  for  J73AVS  are  the  IBM  370,  DEC  20,  and 
VAX  11/780. 
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ABSTRACT 


The  development  of  J73AVS  reflects  the  commitment  of  the  Air  Force 
to  facilitate  JOVIAL  J73  standardization.  This  paper  describes  software 
verification  as  it  is  automated  by  the  J73AVS  tool.  The  concept  of 
software  verification  is  discussed,  as  well  as  the  capabilities  and 
operation  of  J73AVS.  J73AVS  provides  much  of  its  payoff  by  detecting 
certain  software  errors  and  measuring  the  thoroughness  of  testing  far 
more  accurately  and  efficiently  than  could  be  achieved  manually.  While 
J73AVS  operates  as  a  standalone  program  on  several  host  computers,  it 
augments  the  JOVIAL  J73  support  environment  when  used  with  other  Air 
Force-sponsored  tools  such  as  the  code  auditor  [1],  debugger  [2),  and 
Program  Support  Library  [3]. 


INTRODUCTION 


What  is  "software  verification"?  According  to  the  IEEE  committee 
on  standardizing  software  terminology,  it  is 

"the  iterative  evaluation  of  evolving  software  to  ensure 
compliance  with  requirements" 

Thus,  verification  differs  from  validation  or  certification  in  that  it 
is  an  activity  that  is  performed  continuously  throughout  a  software 
development  cycle.  It  incorporates  a  variety  of  automated  and  manual 
techniques  to  determine  consistency  between  the  requirements,  design, 
coding,  testing,  and  documenting  stages  of  software  projects. 

Our  focus  in  designing  and  building  Automated  Verification  Systems 
(AVS)  for  JOVIAL  [4],  FORTRAN  (5),  and  COBOL  [6]  has  been  on  static  and 
dynamic  code  analysis.  That  is,  each  AVS  reads  source  code  as  input  for 
static  analysis  and  uses  the  program's  regular  input  data  during  dynamic 
analysis.  Therefore,  requirements  and  design  are  not  verified  directly 
by  the  AVS.  However,  because  the  AVS  analyzes  the  actual  code,  the  tool 
reports  true  program  characteristics.  This  approach  interferes  very 
little  with  normal  program  development,  since  the  AVS  does  not  require 
additional  information  other  than  the  source  code  and  initial  set  of 
test  data. 
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One  very  important,  but  often  neglected,  area  of  software  veri¬ 
fication  is  that  of  ensuring  that  the  program  documentation  reflects  the 
real  code.  The  most  time-consuming  aspect  of  conforming  to  Mil-Std-483 
or  other  documentation  standards  is  generating  program  symbol,  struc¬ 
ture,  and  interface  information.  Because  it  maintains  a  database  of 
intra-  and  inter-module  characteristics,  the  AVS  can  generate  some  of 
this  information  automatically.  As  code  is  modified  due  to  error 
correction  or  enhancements,  the  reports  can  be  easily  regenerated  to 
reflect  the  changed  code.  In  contrast,  manually  generated  documentation 
is  rarely  up  to  date. 

J73AVS  CAPABILITIES 

J73AVS  should  play  a  role  in  JOVIAL  J73  software  development  as 
soon  as  some  of  the  modules  are  compilable.  The  source  code  is  gener¬ 
ated  based  upon  a  design,  which  in  turn  is  based  on  a  set  of  require¬ 
ments.  It  is  recommended  that  the  expected  program  output  (acceptance 
criteria)  and  at  least  an  initial  set  of  input  test  data  be  generated 
concurrently  with  the  program's  design.  The  requirements,  design,  and 
acceptance  criteria  play  an  indirect  role  in  J73AVS's  analysis  of  the 
software. 

The  types  of  J73AVS  analysis  capabilities  are: 

•  Static  and  data-flow  analysis  (symbol  usage  anomalies  and 
dangerous  coding) 

•  Reporting  of  program  structure  and  characteristics 

•  Measuring  execution  coverage  of  statements,  branches,  and 
program  units 

•  Execution  tracing  of  variables,  branches,  and  program  units 

•  Execution  timing 

•  Structural  (branch)  retesting  assistance 

•  Test  history  reporting 


Figure  1  shows  how  the  requirements,  design  specification, 
acceptance  criteria,  and  test  data  interact  with  J73AVS-supported 
testing.  The  acceptance  criteria  are  used  to  judge  the  proper  perform¬ 
ance  of  the  program.  J73AVS  provides  detailed  source  analysis  reporting 
which  aids  the  analyst  in  determining  that  certain  acceptance  criteria 
are  being  met.  The  bold  path  marked  number  one  in  Figure  1  indicates 
the  cycle  of  static  code  checking. 


Figure  1.  Using  an  AVS 


Once  the  static  errors  are  removed,  the  program  can  be  analyzed 
dynamically  by  driving  it  with  the  initial  set  of  test  data.  Dynamic 
analysis  is  indicated  by  bold  path  number  two  in  Figure  1.  J73AVS 
outputs  execution  coverage,  timing,  and  tracing  information,  along  with 
the  normal  program  output,  which  aids  the  analyst  in  determining 
acceptable  performance.  Unexercised  statements  and  branches  are 
indicated  by  J73AVS  so  that  additional  test  data  can  be  generated  to 
ensure  that  all  parts  of  the  code  are  tested.  Dynamic  analysis, 
therefore,  is  usually  an  iterative  activity  that  continues  until  the 
desired  level  of  exercise  is  achieved.  J73AVS  maintains  the  coverage 
levels  for  each  test  in  its  database. 

As  shown  in  Figure  1,  compilable  source  code  generally  is  first 
analyzed  by  J73AVS  to  detect  semantic  errors  that  are  outside  the  scope 
of  the  compiler's  static  analysis  capabilities.  As  each  module  is 
analyzed  by  J73AVS,  a  database  is  built  that  contains  single  and 
multi-module  detailed  characteristics.  This  database  is  used  and 
augmented  each  time  additional  analysis  (static  or  dynamic)  is  requested 
by  the  user.  Thus,  J73AVS  is  a  partner  in  the  development,  testing,  and 
documentation  phases. 
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It  should  be  pointed  out  that,  because  of  the  database  feature, 
J73AVS  supports  top-down  code  development  in  the  following  way. 
High-level  modules  can  be  coded  early  with  stubs  (module  skeletons)  for 
lower-level  modules.  Both  fully  coded  modules  and  stubs  can  be  input  to 
J73AVS  for  analysis  and  documentation.  J73AVS  Includes  the  stubs  in  its 
database.  Module  interaction  and  interfaces  (COMPOOL  usage  and  para¬ 
meter  passing)  will  be  analyzed  and  reported  to  the  extent  that  they 
occur  in  the  code.  Then,  as  lower-level  stubs  are  replaced  with  full 
source  code,  J73AVS  replaces  the  modules  on  the  database. 


A  typical  sequence  of  J73AVS-supported  verification  of  fully  coded 
source  modules  is : 


JOVIAL  J73  source  text,  perhaps  with  assertions  (Boolean 
expressions,  recognized  by  J73AVS,  that  specify  expected 
behavior),  is  read  by  J73AVS  as  one  or  more  compilable 
modules. 


J73AVS  produces  program  analysis  reports  showing  control 
structure,  symbol  usage,  calling  hierarchy,  etc.,  as  well  as 
a  static  analysis  report  showing  errors  and  dangerous 
programming  practices. 


Using  the  reports  as  a  guide,  the  source  modules  are  changed 
or  new  modules  are  added  to  the  program. 


J73AVS  reports  the  interaction  of  the  new  or  changed  modules 
with  the  rest  of  the  program.  This  information,  in  turn, 
may  show  the  need  to  modify  other  modules. 


For  debugging,  the  program  is  instrumented  by  J73AVS  and 
executed  with  an  initial  test  case  supplied  by  the  user. 


Assertion  messages,  variable,  branch,  and  module  tracing, 
and  execution  timing  reports  can  be  used  for  debugging. 


Using  the  J73AVS  reports,  the  user  chooses  to  create  more 
test  data  or  instrument  other  modules. 


For  testing,  the  same  cycle  of  instrumentation  and  execution 
is  repeated,  but  for  a  different  goal:  rather  than  detecting 
and  locating  errors,  testing  aims  to  demonstrate  that  the 
entire  program  has  been  exercised  to  some  degree.  The 
J73AVS  execution  analysis  reports  show  the  thoroughness  of 
execution  coverage. 


The  user  evaluates  execution  coverage  reports,  the  program's 
own  execution  results,  and  the  program  specification  to 
determine  if  testing  is  complete. 
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J73AVS  provides  branch  sequence  information  to  retest 
targets  chosen  by  the  user.  A  test  history  of  execution 
coverage  assists  the  user  in  choosing  targets  for  retesting 


As  just  noted,  J73AVS  can  be  used  to  assist  with  several  phases  of 
software  development.  These  phases  can  be  grouped  as: 

•  Program  development  and  maintenance 

•  Debugging 

•  Testing 

•  Retesting  assistance 


Program  Development  and  Maintenance 

Executable  assertions  provide  a  means  for  a  programmer  to  specify 
expected  behavior.  Assertions  can  be  used  for  reporting  execution- time 
exceptions,  stress  testing,  and  manual  or  automated  test  data  genera¬ 
tion.  When  assertions  are  left  as  comments  in  the  source  code  they  can 
be  used  as  inline  documentation  of  the  program’s  specifications.  An 
example  of  an  executable  assertions  is: 

".  ASSERT  (STACK' POINTER  >«  0)” 

To  assist  with  reliable  system  development  and  maintenance,  J73AVS 
provides  substantial  program  analysis  reporting  on  structural  hierarchy, 
symbol  usage,  invocations,  certain  J73  constructs,  and  system  character¬ 
istics.  The  user  has  control  over  obtaining  high-  or  low-level  infor¬ 
mation  through  the  tool's  command  language. 


Debugging 

Normal  compilation  using  JOVIAL  J73  compilers  can  detect  many 
syntax  and  semantic  errors.  Other  errors,  such  as  uninitialized 
variables,  possible  Infinite  loops,  unreachable  code,  certain  improper 
constructs,  and  dangerous  coding  practices  (like  transferring  into  CASE 
or  IF  statements)  will  be  reported  by  J73AVS.  The  user  can  specify  the 
degree  of  analysis  to  the  error,  warning,  or  message  level. 

Debugging  is  supported  by  assertion  exceptions,  variable  and 
module  execution  tracing,  and  execution  timing  reports.  When  the 
program's  execution  behavior  deviates  from  the  acceptable  logical 
behavior  as  specified  by  the  assertions,  it  is  immediately  reported  in 
the  program's  output.  The  user-embedded  assertions  have  no  effect  on 
program  control  flow  until  they  are  violated;  at  that  time  the  violation 
is  reported  with  the  source  statement  number  of  the  assertions. 


Testing 

The  primary  purpose  of  program  coverage  analysis  is  to  provide  a 
measure  of  the  level  of  testing.  One  measuring  technique  uses  the 
program's  control  structure  as  a  guide.  Structure-based  testing  means 
that  the  program's  control  structures  are  analyzed  for  execution 
behavior;  that  is,  whether  che  structures  are  exercised  and  in  what 
order.  Structure-based  testing  can  uncover  errors  due  to  untested 
branches  or  improper  sequences  cf  branches.  J73AVS  provides  program 
unit  or  branch  tracing  and  analyzes  execution  coverage  of  program  units, 
branches,  and  statements.  Further,  J73AVS  assembles  the  timing  infor¬ 
mation  from  program  unit  tracing  and  user-directed  timing  probes  into  an 
execution  timing  report. 


Retesting  Assistance 

Software  is  retested  when  analysis  shows  that  prior  testing  is 
inadequate  (insufficient  branch  coverage,  not  all  functions  demon¬ 
strated,  etc.)  or  when  program  changes  have  taken  place.  The  proper 
approach  to  take  in  retesting  is  highly  dependent  upon  the  character¬ 
istics  of  the  program  being  tested  as  well  as  the  measures  being  used  to 
evaluate  testing  completeness. 

To  determine  the  sequence  of  branches  which  lead  to  an  untested 
branch  or  statement,  the  user  can.  request  that  the  "reaching  set"  be 
computed  between  two  specified  statements  (or  from  a  procedure's  entry). 
After  the  flow  of  control  is  identified  by  J73AVS,  the  user  can  back¬ 
track  through  the  program  to  the  actual  test  data.  New  test  data  can  be 
created  by  using  J73AVS  module  Interaction,  invocation,  and  execution 
coverage  reports.  Unfortunately,  automatic  test  data  generators  which 
use  symbolic  execution  are  not  yet  general  enough,  easy  to  use,  or 
reliable.  Therefore,  J73AVS  has  no  test  data  generation  capability  at 
this  time. 

The  testing  history  maintained  by  J73AVS  is  useful  in  attaining 
testing  coverage  goals  and  for  determining  targets  for  retesting. 
Procedure  invocation  and  coverage  information  is  saved  in  a  concise  way 
for  each  test  case.  The  results  of  subsequent  execution  runs  can  be 
added,  providing  a  cumulative  report  of  all  tests. 


J73AVS  OPERATION 

J73AVS  operates  in  either  batch  or  interactive  mode  on  a  host 
computer.  If  the  JOVIAL  J73  code  being  analyzed  is  destined  for 
execution  on  a  target  computer,  the  J73AVS  dynamic  analysis  operation  is 
modified  slightly,  as  shown  in  Figure  2. 


Figure  2.  Using  J73AVS  in  a  Host-Target  Environment 


The  user  directs  J73AVS  analysis  through  a  simple  command  lan¬ 
guage.  The  basic  commands  are  verbs  that  select  the  type  of  analysis, 
followed  by  command  parameters  that  specify  the  scope  of  the  analysis  or 
level  of  error  reporting.  As  much  as  a  whole  program  or  as  little  as  a 
single  symbol  can  be  analyzed. 

J73AVS  displays  or  prints  reports  during  static  analysis.  For 
dynamic  analysis,  the  instrumented  source  (augmented  by  expanded 
assertions  and  by  probes  for  execution  coverage,  tracing,  or  timing)  is 
passed  to  the  JOVIAL  J73  compiler.  In  a  host  execution  environment, 
input  data  is  read  by  the  program  and  normal  program  output  is  accom¬ 
panied  by  an  execution  data  collection  file  required  for  the  J73AVS 
post-execution  analysis  reports.  J73AVS  uses  that  file,  along  with  its 
database,  to  provide  readable,  user-selected  reports  that  describe 
execution  behavior. 

In  a  host-target  environment,  the  target  computer  must  have  a 
sequential  output  device  such  as  a  disk  or  tape  to  transfer  the  data 
collected  during  execution  back  to  the  host.  In  the  absence  of  any 
sequential  output  device,  the  J73AVS  data  collection  routines  can  be 
modified  to  output  test  coverage  information  on  the  target  in  an 
abbreviated  manner. 


J73AVS  was  developed  for  operation  on  IBM  370  and  DEC  20  compu¬ 
ters.  It  is  currently  being  rehosted  to  the  VAX  11/780.  J73AVS  is 
written  in  JOVIAL  J73,  except  for  a  few  small  input/output  routines 
written  in  FORTRAN.  The  VAX  version  of  J73AVS  is  written  in  a  struc¬ 
tured  dialect  of  FORTRAN. 
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APPENDIX  B 

TERMS  AND  ABBREVIATIONS 


The  following  terms  pertain  to  software  verification  and  to  the  charac¬ 
teristics  of  JOVIAL  software. 

Assertion  - 

Statement  of  a  condition  that  must  be  true  whenever  control 
reaches  that  point  in  the  program. 

AVS  - 

Automated  Verification  System.  A  computer  program  or  collection 
of  programs  which  assists  ic  verifying  the  correspondence  between 
software  and  the  set  of  functional  specifications  defining  the 
software . 

Branch  - 

A  continuously  executable  sequence  of  statements  between  two 
decision  statements.  It  may  include  unconditional  transfers. 

Branch  testing  - 

A  testing  technique  that  measures  the  number  of  executions  of  each 
branch  in  a  module  and  computes  a  measure  for  testing  thoroughness 
in  terms  of  the  fraction  of  all  branches  executed  at  least  once. 

Data  Flow  Analysis  - 

A  program  analysis  technique  which  tracks  the  usage  of  symbols 
through  a  program.  The  technique  is  used  to  detect  uninitialized 
variables  and  other  program  anomalies  based  on  symbol  usage  and 
control  flow.  The  program  is  not  actually  executed. 

Dynamic  Analysis  - 

A  debugging  or  testing  technique  in  which  the  program  is  executed 
with  data  and  the  program  output,  together  with  any  additional 
execution-time  reports,  is  analyzed  for  conformity  to  functional 
or  structural  performance  specification. 
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Instrumentation  - 


The  technique  of  automatically  inserting  software  probes  (sub¬ 
routine  calls  or  counters)  into  code  or  of  translating  special 
statements  (like  an  assertion)  into  executable  code  for  the 
purpose  of  collecting  information  during  program  execution. 


Interface  Description  - 


Information  in  the  AVS  describing  global  data  passed  between 


modules . 


J73AVS  - 


An  Automated  Verification  System  for  JOVIAL  J73  Software, 


JOVIAL  J73  - 


A  JOVIAL  programming  language  as  defined  in  MIL-STD-1589B  for 
command  and  control,  avionics,  and  defense  systems  applications. 


Module  - 


The  smallest  entity  in  JOVIAL  J73  that  can  be  separately  compiled. 
(Note  the  difference  between  module  and  program  unit.) 


Path  - 


A  continuous  sequence  of  control  flow  (branches)  between  two 
points  in  a  program  (usually  between  a  program  unit's  entry  and 
exit) . 


Program  Unit  - 


The  smallest  entity  in  JOVIAL  J73  that  can  be  invoked,  or,  in  the 
case  of  compools,  referenced  by  name.  In  JOVIAL  J73  program  units 
are  compool-modules ,  main-programs,  procedures,  and  functions. 


Reaching  Set  - 


A  set  of  statements  that  incorporates  all  of  the  branches  leading 
to  a  specified  statement. 


Static  Analysis  - 


A  technique  which  analyzes  program  source  but  does  not  actually 
execute  the  program.  Program  statements  and  symbols  are  analyzed 


to  detect  inconsistencies  in  semantics  or  in  asserted  versus 


actual  conditions. 
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Test  Target  Selection  - 

The  process  of  selecting  a  module,  or  a  set  of  statements  within 
module,  to  be  executed  with  data  for  the  purpose  of  improving  the 
quality  of  testing.  Goals  for  improving  the  quality  of  testing 
may  be  exercising  unexecuted  branches,  paths,  or  statements, 
executing  all  boundary  conditions,  etc. 


J73AVS  Processing  and  Reporting  Commands 


Command  (Defaults  Underlined) 

READ  { .ECHO} 

STATIC  {.DATA}  { .SUMMARY /FULL} 

{ ,LIST°ERRORS .WARNINGS .MESSAGES} 


LIST 

LIST .DATABASE  {.UNITS} 

INSTRUMENT  {  .ASSERTIONS}  { .COVERAGE}  { , TRACE* BRANCH/ ENTRY } 
INSTRUMENT ,NEWTEST,<m-name> ,<stmt> 

INSTRUMENT .CLOCK, ON=<m-name> ,<stmt> ,OFF»<m-name> ,<stmt> 
INSTRUMENT .VARIABLE ,<v-name>  { ,<start-stmt>} ,{<stop-stmt>} 
INSTRUMENT, GO. 

DOCUMENT, INVOCATIONS  { .SOURCE}  { .BANDS /BANDS*<n>} 

{.TREE}  {.GLOBAL} 

DOCUMENT, SYMBOLS  { .SOURCE/XREF}  { ,NAMES*<s-name> , . . . } 
DOCUMENT .LABELS/TYPES/CNSTANTS/COMPOOLS/REFDEF  { .SOURCE/XREF} 
{ ,NAMES=<namel> , . . . } 

DOCUMENT .DEFINES  { .XREF/EXPANSIONS/FULL}  { ,NAMES=<namel> , . . . } 

EXECUTION  {.SUMMARY}  {SOURCE}  { .NOTHIT}  {.TIMING} 

EXECUTION, GO 

ASSI ST, BRANCHES { { ,<first  stmt>} ,<last  stmt>{ .ITERATIVE}} 
ASSIST .HISTORY { .RESET} 


note:  {}  indicates  optional  parameter 

<>  indicates  user-supplied  name 
/  indicates  selection  of  one  parameter 


All  command  keywords  and  parameters  can  be  abbreviated  to  two  or  more  letters 


RA VC  plant  and  executes  research,  development,  tut  and 
selected  acquisition  programs  In  support  oi  Command,  Control 
Communications  and  Intelligence  IC*J)  activities.  Technical 
and  engineering  support  coithin  areas  otf  technical  competence 
is  provided  to  ESP  Program  0 Mitts  IPO*)  and  other  BSD 
elements.  The  principal  technical  mission  areas  oat 
communications ,  electromagnetic  guidance  and  control,  sur¬ 
veillance  oi  ground  and  aerospace  objects,  intelligence  data 
collection  and  handling ,  information  system  technology, 
ionospheric  propagation,  solid  state  sciences,  micro wave 
physics  and  electronic  reliability,  maintainability  and 
compatibility. 


